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The  purpose  of  this  study  was  to  develop  and  test  a  hypertext 
version  of  the  Certified  Professional  Contracts  Manager  Candidate's 
Md  Sypslsmen^t. 

A  literatfire  review  was  conducted  on  two  distinct  subject  areas. 
The  first  area  includes  a  review  of  the  findings  of  six  conmissions  on 
professionalism  of  acquisition  personnel,  initiatives  of  the  Office  of 
Personnel  Management  (0PM),  the  Defense  Acquisition  Workforce  Improve* 
ment  Act,  and  the  National  Contract  Management  Association.  The 
discussion  of  professionalism  in  contracting  establishes  the  relevance 
of  products  like  the  hypertext  workbook  completed  as  a  result  of  this 
research.  The  second  area  is  a  review  of  the  history  of  hypertext, 
applicable  definitions,  application  schemes,  and  system  design. 

Through  a  development  effort  that  followed  five  objectives,  a 
coiiq>lete  hypertext  workbook  was  designed  and  tested.  The  results  of 
the  questionnaire  answered  by  ten  graduate  students  revealed  that  the 
hypertext  workbook  was  easy  to  install,  learn  and  use. 
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A  HYPERTEXT  VERSION  OF  THE  CERTIFIED  PROFESSIONAL 
CONTRACTS  MANAGER  CANDIDATE'S  WORKBOOK 

I.  Introduction 

This  section  introduces  the  research.  First,  the  growing 
ioportance  of  continued  education  within  federal  contracting  is 
discussed.  Second,  a  brief  discussion  of  the  National  Contract 
Management  Association  (NCMA)  is  presented.  NCMA's  certification 
programs  are  explained,  in  addition  to  the  Certified  Professional 
Contracts  Manager  Candidate's  Workbook.  Third,  the  disadvantages 
associated  with  the  linear  version  of  the  workbook  ere  described. 

Fourth  is  a  concise  description  of  hypertext  as  it  will  be  used  for  the 
development  of  a  computerized  CPCM  Candidate's  Workbook.  Fifth,  the 
objectives  that  structure  this  research  are  presented.  Sixth,  the 
scope  and  limitations  of  this  research  are  identified.  The 
organization  of  this  thesis  is  outlined  last. 

The  Contracting  Work  Force 

The  contracting  work  force  is  professionalizing.  Since  1950,  six 
major  connissions  have  made  reconmendations  related  to  heightened 
professionalism  in  the  procurement  arena.  More  recently,  the  Defense 
Acquisition  Workforce  Improvement  Act  (DAWIA),  H.R.  4739,  carried  out 
the  recommendations  of  earlier  commissions. 

DAWIA  was  attached  to  the  Department  of  Defense  Authorization  Act 
for  Fiscal  Year  1991.  It  focuses  on  the  education,  training,  and 
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carMr  development  of  both  the  military  and  civilian  con^onents  of  the 
federal  contracting  work  force.  Included  in  the  legislation  are  plans 
to  form  an  Acquisition  Corps.  DoD  Directive  5000.52  defines  the 
Acquisition  Corps  as: 

A  narrowly  defined  part  of  a  Service's  acquisition 
workforce,  coiiq>osed  of  selected  military  and  civilian 
acquisition  personnel  who  dedicate  their  careers  as 
acquisition  specialists.  (8:v) 

Several  criteria  must  be  met  in  order  for  an  individual  to  be 
recognized  as  a  member  of  this  elite  group  of  professionals.  Among  the 
prerequisites  are  educational  standards.  The  future  of  professional 
continuing  education  for  this  group  is  therefore  growing  in  importance. 

In  order  to  keep  pace  with  these  increased  standards  and 
challenges,  individuals  can  attend  one  of  the  over  100  colleges  and 
universities  that  offer  undergraduate  and  postgraduate  contract 
management  programs  and  courses  (30:23).  In  addition,  professional 
organizations  exist  that  can  assist  in  meeting  these  needs. 

The  National  Contract  Management  Association 

The  National  Contract  hanagement  Association  (NCMA)  is  the 
foremost  professional  organization  in  the  contracting  career  field. 
According  to  the  1990  National  President,  James  B.  Lessig,  "Education 
is  our  primary  mission  and  the  most  important  way  we  serve  our  members" 
(22:2).  Record  numbers  are  seeking  professional  certification  through 
the  two  nationally  recognized  certification  programs  offered  by  NCMA. 
The  two  programs  are  the  Certified  Professional  Contracts  Manager 
(CPCM) ,  and  the  Certified  Associate  Contracts  Manager  (CACM) .  Although 
both  programs  are  notable,  the  CPCM  is  the  more  prestigious.  The  "CM 
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Final  Edition"  in  the  January  1991  issue  o£  Contract  Management 
recorded  that  "The  number  of  individuals  sitting  for  the  November  9, 
1990,  certification  exams  -  201  CPCM,  19  CACM  -  surpassed  last 
November's  record  of  173  and  15,  respectively"  (6:34). 

NCMA  educational  goals  are  structured  to  sieet  the  growing  needs 
of  the  procurement  work  force.  The  long-range  goals  for  its  education 
program  are: 

(1)  Preparation  of  educational  materials  in  contract 
manf^ement  based  on  high  standards  of  scholarship  and 
research,  and  (2)  preparation  of  monographs,  reference 
materials,  or  manuals  that  have  value  not  only  in 
university  programs,  but  as  trade  publications  useful  to 
contract  management  practitioners.  (26:57) 

NCMA's  annual  National  Education  Seminar  has  evolved  into  a  well 
recognized  avenue  for  organizations  to  supplement  their  training  needs. 
In  addition,  NCMA  offers  a  wide  variety  of  educational  products  in 
eight  categories:  1)  Topical  Issues  in  Procurement  Series  (TIPS); 

2)  Reference/Study  Materials;  3)  Workshops;  4)  Computer  Diskettes; 

5)  Books;  6)  National  Education  Seminar  Manuals;  7)  Challenge  Monograph 
Series;  and  8)  Videotapes  (22:60). 

Reference/Studv  Materials 

Included  in  the  Reference/Study  Materials  are  the  CPCM 
Candidate's  Workbook,  the  CPCM  Workbook.  Supplement  1.  Supplement  2. 
and  Supplement  3.  (hereafter  referred  to  as  the  Workbook,  and 
Supplements) .  The  four  publications  were  compiled  and  edited  by 
Frederic  M.  Denny,  MBA,  CPCM.  The  purpose  of  the  texts  is  to  help  CPCM 
candidates  organize  an  approach  to  the  CPCM  examinations,  and  to 
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provide  the  contents  of  past  examinations  used  for  preparing  of  future 
tests  (7:xv). 

The  saterials  are  well  planned  and  structured.  They 
unquestionably  provide  the  services  that  were  intended.  The 
publications'  usefulness  is  highlighted  by  an  appendix  that  provides  an 
analysis  of  past  exams.  The  appendix  consists  of  six  different 
gumnaries  that  allow  the  CPCH  candidate  to  review  the  makeup  of  past 
exams.  For  example,  the  second  sumnary  gives  a  sequential  listing  of 
all  of  the  questions  from  all  of  the  exams.  It  groups  the  questions  by 
the  category  in  which  it  first  appeared,  and  includes  the  total  number 
of  appearances  (7:xv).  The  valud  of  the  text  notwithstanding,  the 
Workbook  does  have  certain  disadvantages. 

Disadvantages  of  the  Workbook.  The  disadvantages  of  the 
documents  are  related  to  their  growing  size.  The  Workbook  and 
Supplements  currently  conqprise  over  800  pages.  In  consideration  of 
publishing  and  postage  costs  alone,  it  is  easy  to  see  why  the  three 
materials  cost  $55  for  NCMA  members.  Candidates  must  flip  pages  and 
cross-reference  question  numbers  to  page  numbers  in  the  search  for  a 
question  of  interest.  Since  the  Workbook  alone  consists  of 
approximately  500  pages  filed  in  a  three-ring  binder,  it  can  be 
ctimbersome  to  use.  While  NONA  offers  a  few  of  their  products  in  the 
form  of  computer  diskettes,  nobody  has  explored  the  feasibility  of  an 
electronic  form  of  the  Workbook. 

Advances  in  software  technology  have  made  an  electronic  form  of 
the  Workbook  possible.  The  Executive  Director  of  the  National  Contract 
Management  Association  believes  that  an  "interactive  computerized  study 
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tool  based  on  NCMA's  CPCM  Candidate's  Workbook  and  its  supplements 
would  be  a  major  contribution  to  NCMA's  Educational  Product  offer¬ 
ings”  (14). 

Hypertext  is  one  approach  that  is  bringing  publications  into  the 

information  age.  It  is  a  feature  of  fairly  new  software  programs  that 

makes  possible  the  ability  to  link  information  electronically.  One 

software  manufacturer  described  hypertext  in  the  following  manner: 

In  a  hypertext  document,  rather  than  turning  pages  to  find 
more  information  or  using  an  index  to  find  a  topic,  it  is 
structured  so  that  wherever  more  information  exists  it  is 
linked  off  a  key  word  or  phrase.  (29:1) 

^  Advantages  of  Hypertext  Documents.  A  hypertext  document  can 
potentially  overcome  the  disadvantages  associated  with  bulky 
publications.  Hypertext  gives  readers  no  preconceptions  regarding  the 
size  of  a  document  (28:23).  The  sheer  physical  size  of  the  Workbook 
and  Supplements  might  possibly  intimidate  CPCM  candidates,  or  even 
scare  them  from  taking  the  examination.  "Highly  organized  or 
centralized  hypertext  documents  with  a  very  rigid  structure  tend  to 
seem  smaller  than  their  actual  length"  (28:23).  Upon  entering  a 
hypertext  doc'.tment,  users  can  choose  from  a  number  of  desired  paths 
toward  an  area  of  interest.  The  process  of  browsing  information  is 
systematic  and  focused,  while  the  length  of  the  document  is  less 
obvious  (28:23).  Hypertext  systems  eliminate  the  flipping  of  pages; 
cross-referencing  takes  less  than  a  second.  A  hypertext  workbook  could 
also  be  supplemented  periodically.  Additionally,  the  costs  associated 
with  mailing  diskettes  would  be  substantially  less  than  heavy  paper 
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The  purpose  of  the  research  was  to  develop  and  test  a  hypertext 
version  of  the  CPCM  Candidate's  Workbook  and  Supplements.  The  Workbook 
is  functional;  however,  the  stated  potential  benefits  of  a  hypertext 
version  exist,  and  would  facilitate  the  learning  process. 

&eigfEshJ>t>igc,tiYga 

Five  objectives  structured  the  research  effort. 

Obiective  1.  The  first  objective  was  to  obtain  permission  to 
create  a  hypertext  version  of  the  National  Contract  Managentent 
Association's  CPCM  Candidate's  Workbook  and  Supplements . 

Obiective  2.  The  second  objective  was  to  identify  the  important 
features  of  hypertext  authoring  systems  needed  for  the  research. 

Obiective  3.  The  tnird  objective  was  to  develop  the  hypertext 
SKxlel.  Three  sub-objectives  structured  this  step  even  further. 

Sub-Ob iective  3a.  The  first  sub-objective  was  to  design 
the  hypertext  model's  structure  on  paper. 

Sub-Obiectivc  3b.  The  second  sub-objective  was  to  convert 
the  text  of  the  Workbook  and  Supplements  into  machine  readable  format. 

Sub-Ob  iective  3c.  The  third  sub-objective  was  to  determine 
a  method  with  which  to  present  the  information  within  the  hypertext 
system. 

Obiective  4.  The  fourth  objective  was  to  test  and  evaluate  the 
hypertext  system. 

Obiective  5.  The  fifth  objective  wan  to  implement  improvements 
to  the  hypertext  system  that  were  identified  during  the  testing  stage. 


Software  development  requires  several  iterations  of  the  above 
objectives  to  refine  a  system  to  its  fullest  potential.  This  research 


involved  only  one  iteration. 

Potential  users  of  the  hypertext  system  include  a  vast  population 
of  civilian  and  military  contracting  personnel  across  the  country.  A 
comprehensive  review  of  the  customers'  needs  was  beyond  the  scope  of 
the  research.  An  initial  model  was  developed  to  test  the  feasibility 
of  a  hypertext  version  of  the  Workbook .  and  to  receive  initial  feedback 
of  the  model.  To  facilitate  the  process,  an  Air  Force  Institute  of 
Technology  faculty  member  and  fellow  graduate  student  acted  as  the 
users.  Both  are  CPCM's,  and  the  faculty  member  is  an  honorary  life 
member  of  the  National  Contract  Management  Association. 

Thesis  Outline 

Chapter  1  introduces  the  research  by  presenting  a  background  of 
the  specific  problem  and  identifying  the  research  objectives. 

Chapter  2  expands  on  the  relevant  topics  of  the  research, 
development  of  professionalism  in  contracting  from  an  historical 
perspective,  and  hypertext  systems. 

Chapter  3  is  a  discussion  of  the  methodology  that  was  used  to 
develop  and  test  the  hypertext  document. 

Chapter  4  reports  the  results. 

Chapter  5  coiig>letes  the  research  with  conclusions  and 
recotonendations . 
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This  chapter  includes  discussions  of  two  dissimilar  subject 
areas.  They  are  professionalism  in  contracting  from  a  historical 
perspective  and  issues  regarding  hypertext. 

The  first  major  discussion  begins  with  the  findings  and 
recosnendations  of  six  major  coomissions  on  professionalism  in 
contracting.  Initiatives  since  the  Defense  Management  Review  of  1989 
are  then  examined,  followed  by  a  short  discussion  of  education  in 
contracting.  Next  is  a  description  of  the  National  Contract  Management 
Association  (NCMA),  its  membership,  mission,  certification  programs, 
body  of  knowledge,  and  endeavors  to  in^rove  the  body  of  knowledge. 

The  second  major  topic  includes  issues  regarding  hypertext. 

First,  the  history  and  definition  of  hypertext  will  be  given.  Other 
key  terms  associated  with  the  topic  will  be  defined.  Hypertext 
application  schemes  will  be  discussed  third.  Fourth,  the  steps 
required  to  develop  s  hypertext  system  will  be  explained.  Finally, 
this  chapter  closes  with  important  issues  and  caveats  regarding  the 
design  of  hypertext  programs. 

Professionalism  in  Contracting 

The  Unit<^  States  Government’s  role  as  a  leading  defender  of 
deiBocracy  has  been  a  costly  endeavor.  In  Fiscal  Year  1988  alone,  the 
Department  of  Defense  (DoD)  contracted  for  over  $161.7  billion  in 
services,  supplies,  and  sulitary  hardware  (37:330).  Thus,  it  is  not 
surprising  that  the  acquisition  of  major  weapon  systems  and  the 
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eoap«tition  of  these  efforts  with  non-defense  programs  have  long  been 
subjects  of  public  scrutiny.  In  fact,  apprehension  regarding  the  use 
of  public  ftmds  for  the  procurement  of  military  supplies  and  equipment 
dates  back  to  the  French  and  Indian  Mars  when  the  price  and  quality  of 
goods  were  issues  of  controversy  (39:2). 

Since  1950,  seven  major  comnissions  have  been  established  by 
either  the  President  or  Congress  to  review  U.S.  defense  acquisition 
organization  and  procedures.  The  reconmendations  provided  by  these 
reports  address  a  number  of  similar  issues  (39:4).  In^roving  the 
quality  of  the  acquisition  work  force  is  a  theme  of  six  of  these 
comnissions,  and  will  be  discussed  further. 

Six  Maior  Comnissions 

The  following  discussion  is  limited  to  the  findings  and 
recommendations  of  six  comnissions  on  professionalism  of  acquisition 
personnel.  The  word  acquisition  is  used  in  the  broad  sense,  and 
includes  functions  such  as:  program  management;  systems  planning, 
research,  development,  and  testing;  contracting;  engineering; 
manufacturing  and  production;  property  management;  logistics;  quality 
control  and  assurance;  and  several  others.  The  commissions  include: 

1)  two  Hoover  Comnissions  (1949  and  1955);  2)  the  Fitzhugh  Commission 
(1970);  3)  the  Grace  Comnission  (1983);  4)  the  Packard  Commission 
(1986);  and  5)  the  Defense  Management  Review  (1989). 

First  Hoover  Copia.ission  (1949).  The  First  Hoover  Comnission  was 
performed  shortly  after  the  National  Security  Act  of  1947,  and  thus 
concentrated  on  broad  organizational  issues.  However,  traces  of 
interest  regarding  defense  procurement  can  be  found  in  the  task  force 
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working  paper  on  national  security  organization.  It  addressed  the  need 
for  increased  con^etency  and  frugality.  The  following  consent  from  the 
executive  aunnary  typifies  the  sustained  public  concern  over 
procurement : 

Nowhere  is  cost-consciousness  more  essential  than  in  the 
ulitary  establishment  -  that  vast  and  complex  organization 
which  currently  absorbs  more  than  30  percent  of  the  annual 
national  budget.  (39:6) 

Second  Hoover  Commission  (195S).  The  Second  Hoover  Cosmission 

specifically  addressed  the  failure  of  Secretary  of  Defense  Directive 

4000.8.  This  directive  was  issued  in  1952  to  enrich  the  competence  of 

procurement  personnel  (39:9).  The  following  recosnendatiohs  were  m&de 

at  the  conclusion  of  this  commission  to  rectify  the  lack  of  action: 

1)  Strengthen  career  opportunities.  2)  Establish  definite 
criteria  for  staffing.  3)  Increase  pay  in  the  upper 
grades.  4)  Confine  rotation  of  military  managers  to 
specialized  support  areas;  and  5)  Assign  officers  to 
management  positions  for  longer  periods.  (39:9) 

Pitzhugh  Conmission  (1970).  Fifteen  years  after  the  Second 

Hoover  Conmission,  the  Pitzhugh  Conmission  was  appointed  to  address 

concerns  regarding  cost  overruns  and  the  organization  and  management  of 

the  Department  of  Defense.  Special  attention  was  given  to  DoD's 

procurement  policies  and  practices.  The  following  paragraph  addressed 

the  state  of  the  DoD  procurement  work  force: 

Regardless  of  how  effective  the  overall  system  of 
Department  procurement  regulations  may  be  judged  to  be,  the 
key  determinants  of  the  ultimate  effectiveness  and 
efficiency  of  the  Defense  Procurement  process  are  the 
procurement  personnel  who  have  the  challenging 
responsibility  for  interpreting  and  applying  the 
regulations  and  associated  guidance  material.  The 
importance  of  this  truism  has  not  been  appropriately 
reflected  in  the  recruitment,  career  development,  training, 
and  management  of  the  procurement  work  force.  As  a 
consequence,  the  Department  is  faced  with  a  significant 
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mmber  of  innediate  and  future  problems  with  respect  to  the 
availability  in  adequate  numbers  of  appropriately  qualified 
and  capable  procurement  personnel.  For  example,  major 
problems  exist  with  respect  to  their  aging,  turnover, 
capabilities,  and  utilization.  (39:241) 

Grace  Commission  (19831.  The  Grace  Commission  was  an  executive 
committee  established  by  President  Reagan  in  1982.  The  charter  of  the 
council  was  to  identify  improvement  areas  that  could  reduce  cost.  The 
following  is  a  pertinent  statement  found  in  the  executive  summary  of 
the  report : 

As  would  be  expected  in  a  democracy,  DoD  is  under  constant 
scrutiny  by  Congress,  the  Office  of  Management  and  Budget 
(OMB),  the  White  House,  the  media,  special  interest  groups 
for  and  against  its  activities,  its  employees,  its  retired 
employees,  business  leaders,  and  the  general  public. 

(39:603). 

In  order  to  stand  up  to  the  scrutiny,  the  Grace  Connission  reconinended 
the  development  of  professional  procurement  personnel  (39:VI). 

Packard  Coimission  ( 1986) .  David  Packard  chaired  President 
Reagan's  Blue  Ribbon  Cooinission  on  Defense  Management.  The  Packard 
C.^nmission  reported  on  the  quality  of  defense  acquisition  issues. 
Several  recommendations  in  this  report  were  specifically  designed  to 
enhance  the  quality  of  the  acquisition  work  force.  The  opening  remark 
of  the  section  titled,  "Enhance  the  Quality  of  Acquisition  Personnel", 
begins  with  "DoD  must  be  able  to  attract  and  retain  the  caliber  of 
people  necessary  for  a  quality  acquisition  program"  (39:947).  Furt^ier 
remarks  included. 

Federal  regulations  should  establish  business-related 
education  and  experience  criteria  for  civilian  contracting 
personnel,  which  will  provide  a  basis  for  the 
professionalization  of  their  career  paths.  Federal  law 
should  permit  expanded  opportunities  for  the  education  and 
training  of  all  civilian  acquisition  personnel.  (39:948) 
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eomnissions  toward  the  acquisition  work  force,  little  ground  was  made 
toward  increased  professionalism  in  procurement  during  the  three  years 
after  the  Packard  Commission  (9:13).  Secretary  Cheney's  Defense 
Management  Report  to  the  President  is  a. plan  that  focuses  on  the 
implementation  of  the  Packard  Commission's  reconnendations .  The  report 
specifically  identified  several  steps  needed  to  increase  the 
professionalism  of  the  procurement  work  force.  The  classification  of 
DoD  contracting  officers  as  a  professional  personnel  series  was  one 
rcconmendation  made  for  civilians,  while  plans  for  the  establishment  of 
an  acquisition  corps  of  officers  in  each  Service  were  discussed  for  the 
Bulitary  (9:13-14). 

0PM  Initiatives 

Six  months  after  the  completion  of  the  last  report,  the  Director 

of  the  OPH  signed  a  letter  to  the  Deputy  Secretary  of  Defense  that 

changed  the  classification  of  the  GS-1102,  Contracting  series  from 

administrative  to  professional.  The  letter  stated 

I  have  decided  that  contracting  positions  will  be 
designated  as  professional.  That  designation  recognizes 
the  development  of  the  contracting  occupation  into  one 
which  requires  a  high  level  of  competence  in  several 
disciplines.  In  addition,  OPH  will  issue  soon  a  new 
qtialification  standard  for  contracting  positions.  (27:1) 

Specific  examples  of  occupations  within  the  GS-1102  series  are 

contracting  officers,  administrators,  negotiators,  termination 

specialists,  and  cost/price  analysts.  It  is  difficult  at  this  point  to 

report  the  exact  impact  of  this  change  on  the  work  force;  however,  the 
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change  provides  recognition  that  certain  skills  are  essential  to 
perform  i.'t  the  stated  occupations. 

The  Office  of  Federal  Procurement  Policy  (OFPP,  a  part  of  0MB)  is 
required  by  statute  to  promote  Government-wide  career  management 
programs  for  a  professional  procurement  work  force  (3:1).  To  support 
the  move  toward  professionalism  in  the  career  field,  the  office  has 
teamed  up  with  several  agencies  to  develop  a  detailed  Procurement 
Professionalism  Plan.  The  tasking,  named  the  OFPP  Professionalism 
Group,  includes  members  from  selected  Executive  agencies,  the  Federal 
Acquisition  Institute  (FAI),  the  Office  of  Personnel  Management  and  the 
Merit  Systems  Protection  Board  (3:1).  Monthly  meetings  have  been  held 
since  July,  1990  in  order  to  develop  the  Procurement  Professionalism 
Plan.  The  plan  is  to  include  the  following: 

a.  A  coordinated  approach  for  the  recruitment  of 
procurement  interns  by  Federal  agencies  based  on  forecasted 
requirements 

b.  A  program  for  the  training  of  procurement  personnel  in 
those  competencies  identified  as  necessary  for 
professional-level  performance,  after  coordination  with 
Procurement  Executives.  This  training  program  will  ensure 
the  efficient  and  effective  use  of  procurement  education 
and  training  facilities,  and  promote  on-the-job  training 
and  other  methods  to  inculcate  necessary  competencies  in 
the  most  cost-effective  manner; 

c.  Proposals  for  retaining  qualified  professionals;  and 

d.  Goals  for  and  measures  of  success  in  meeting  the 
objective  of  a  better  qualified  procurement  work  force,  as 
well  as  a  system  for  continually  evaluating  the  efficacy  of 
the  procurement  professional  program.  (4:2) 

Included  in  the  minutes  of  the  Group's  meetings  are  discussions 

regarding  FAI  Blueprints  which  may  eventually  become  tl.e  standard  for 

acknowledgement  of  a  competent  GS-1102  (40:2). 


Defense  Acquisition  Workforce  Improvement  Act 

On  November  6,  1990,  the  Department  of  Defense  Authorization  Act 

for  Fiscal  Year  1991  was  signed  by  President  Bush.  Attached  as  an 

amendment  was  the  Mavroules  bill,  now  called  the  Defense  Acquisition 

Workforce  Improvement  Act  (DAWIA).  The  DAWAI  requires  the 

ingtlementation  of  meaningful  provisions  in  the  acquisition  arena.  Some 

of  the  changes  include  the  identification  of  critical  acquisition 

positions  for  the  establishment  of  an  Acquisition  Corps,  and  the 

formulation  of  a  diverse  arrangement  of  educational  and  training 

opportunities  for  acquisition  personnel  (2:1).  New  educational  and 

experience  standards  are  required  for  entry  into  the  Acquisition  Corps, 

which  will  be  phased  into  effect  over  a  three  year  period. 

At  the  direction  of  the  Under  Secretary  of  Defense  for 

Acquisition  (Honorable  John  A.  Betti),  a  DAWIA  Implementation  Board  was 

formed  in  November  1990  to  advise  on  the  execution  of  the  law.  The 

charter  of  the  Board  states 

The  Board  will  oversee  the  implementation  of  these 
requirements  including:  policies  for  defining,  accessing, 
educating,  training,  managing  and  providing  for  career 
development  of  the  defense  acquisition  workforce;  the 
identification  of  critical  acquisition  positions  and 
policies  for  increasing  the  proportion  of  civilians  in 
these  positions;  implementation  of  Acquisition  Corps  in  the 
Services,  OSD  and  the  Defense  Agencies;  policies  for 
rotation  requirements,  mobility  statements,  examinations, 
waivers  and  a  centra'.'zed  job  referral  system;  and 
requirements  for  a  v  y ety  of  education  and  training 
programs  including  a  Scholarship  program  and  the 
development  of  a  structure  for  a  defense  acquisition 
university.  (2:2) 
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Analysis  of  the  Initiatives 


The  new  professional  classification  of  the  GS-1102,  Contracting 
series,  did  not  by  itself  instantaneously  solve  the  previously 
identified  problems  in  the  career  field.  However,  the  Defense 
Acquisition  Workforce  Inq>rovement  Act  did  identify  qualification 
standards  for  the  future.  According  to  Section  1724  of  the  DAWIA,  no 
person  may  be  en^loyed  by  the  Department  of  Defense  after  1  October 
1993  in  the  GS-1102  occupational  series,  or  be  appointed  as  a 
contracting  officer  (for  other  than  small  purchases),  unless  the  person 
has: 

(1)  received  a  baccalaureate  degree;  or  (2)  conq>leted  24 
semester  hour  of  college  study  in  business,  finance, 
quantitative  methods,  or  management-related  subjects;  or 
(3)  passed  an  examination  considered  by  the  Secretary  of 
Defense  to  demonstrate  skills,  knowledge,  and  at  least  24 
semester  credit  hours  in  business,  finance,  quantitative 
methods  or  management-related  subjects.  (38: §1724) 

The  legislation  does  provide  for  certain  exemptions  and  waivers 

to  the  academic  requirements  established  above.  However,  in  light  of 

the  failure  of  past  commission  recommendations,  the  initiatives  made 

over  the  last  year  reflect  profound  change.  Professionalism  in  the 

contracting  career  field  is  quickly  becoming  a  reality  rather  than  a 

reconinendation.  Training,  education,  and  experience  will  be  the  basis 

for  entry  into  the  GS-1102,  Contracting  Series  and  the  Acquisition 

Corps . 

Bducation 

Based  on  the  above  discussion  of  literature,  it  can  reasonably  be 
concluded  that  specialized  education  in  contracting  disciplines  will  be 
of  growing  concern  among  contracting  individuals.  In  his  thesis.  The 
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Identification  of  Contracting  Terms  in  Support  of  the  Body  of 
Knowledge.  Air  Force  Captain  William  Hauf  discussed  the  tendency  of  the 
contracting  comminity  to  focus  on  post-en^loyment  training  and 
education  programs  (17:19).  As  stated  by  OFPP,  and  observed  by  Captain 
Hatif , 


As  a  professional  and  practical  matter,  nearly  all  direct 
training  of  Contracting  personnel  (both  in  the  Government 
and  in  the  private  sector)  is  accomplished  either  throtigh 
on-the-job  experience  or  through  conmercial  or  semi- 
cosnercial  development  programs.  (17:19) 

In  Chapter  One  it  was  stated  that  there  are  institutions  that  provide 

undergraduate  and  postgraduate  contract  management  programs.  The 

George  Washington  University  procurement  and  training  program  has 

existed  since  1968,  while  the  Federal  Acquisition  Institute  has 

fostered  the  development  of  collegiate  offerings  since  1976  (17:20-21). 

The  Air  Force  Institute  of  Technology  (AFIT)  is  now  completing  its 

fourth  offering  of  a  Masters  of  Science  Degree  in  Contracting 

Management.  The  National  Contract  Management  Association  (NCMA)  has 

also  played  an  important  role  in  fostering  professional  development  in 

the  contracting  conmunity.  NCMA  is  the  principal  professional 

assoc:. ation  in  the  career  field. 


National  Contract  Management  Association  (NCMA) 


The  following  paragraphs  are  a  discussion  of  NCMA's  membership, 
Bussion,  certification  programs,  body  of  knowledge,  and  endeavors  to 
improve  the  body  of  knowledge. 

Membership  and  Mission.  Members  of  NCMA  are  employed  in 
contracting  and  related  fields  both  in  private  industry  and  government 
agencies.  Simply  put,  the  missirn  of  NCMA  is  to  encourage,  guide,  and 
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expand  the  professional  development  of  its  members.  NCMA  provides  a 
broad  spectrum  of  opportunities  for  individuals  to  pursue  professional 
development.  A  few  include  snnthly  meetings  that  feature  well  known 
speakers,  subscriptions  to  the  monthly  Contract  Management  magazine, 
and  two  notable  certification  programs.  NCMA  also  offers  a  wide 
variety  of  educational  and  training  publications  and  its  own  code  of 
ethics  (7:v). 

Certified  Professional  Contracts  Mana>tar  (CPCM).  The  CPCM  is  the 
more  prestigious  of  the  two  professional  certification  programs  offered 
by  NCMA.  The  exam  consists  of  two,  three-hour  sessions.  During  the 
first  sitting,  a  candidate  must  answer  five  general  examination 
questions.  The  second  session  requires  five  responses  from  questions 
in  areas  of  concentration  that  correspond  to  contracting  disciplines. 
There  are  a  total  of  seven  areas  of  concentration;  however,  the 
candidate  must  only  answer  three  questions  from  one  area  of  his/her 
choice,  and  one  question  from  each  of  two  other  areas. 

Body  of  Knowledge.  NCMA  made  a  conscious  effort  to  relate  its 

certification  programs  to  a  systematic  Body  of  Knowledge  (7:ix). 

According  to  the  NCMA  Board  of  Directors, 

Professional  proficiency  in  contract  management  requires 
broad  knowledge  and  a  range  of  skills  in  the  areas  of: 

1.  Business  management,  particularly  materials  and 
operations  management,  industrial  marketing,  financial 
management,  and  related  accounting. 

2.  The  economics  of  materials  and  operations  management. 

3.  Cost  and  price  analysis  and  negotiation  techniques. 

4.  Legal  and  regulatory  aspects  of  procurement  and 
contracting. 

5.  Managerial  planning,  decision  making,  comounication, 
and  control . 

6.  Procurement  and  contracting  policy  and  procedures. 
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7.  Management  information  ayatems,  and  information  and 
data  analysia.  (7:s) 

The  Body  of  Knowledge  is  conq;>ri8ed  of  69  separate  blocks  which  fall 

under  three  siajor  categories:  1)  Basic  Tools  and  Functions;  2) 

Contracts  and  Proctirement ;  and  3)  Special  Considerations  (Denny :ix). 

Dr.  David  V.  Lamn  explained  the  significance  of  the  Body  of  Knowledge 

in  the  June,  1985  issue  of  Contractine  Management. 

The  body  of  knowledge  serves  as  the  foundation  of  our 
profession  and  focuses  our  efforts  in  education,  training, 
certification,  research,  writing,  and  the  development  of 
professional  standards.  The  body  of  knowledge  is  necessary 
for  the  practitioner  because  it  identifies  the  functions 
established  in  other  professional  areas.  The  body  of 
knowledge  is  necessary  for  the  theoretician  because  it 
specifies  a  theoretical  framework  which  guides  development 
of  the  fundamentals  and  principles  of  our  profession. 

(21:30) 

Figure  1  represents  NCMA's  Body  of  Knowledge.  The  69  blocks  are  broken 
down  based  on  the  three  categories  discussed  above. 

The  Certified  Professional  Contracts  Manager’s  Handbook,  and  its 
Supplements  give  interested  candidates  a  feel  for  the  scope  of  the 
Professional  Body  of  Knowledge  discussed  above,  in  addition  to 
information  regarding  the  examination  itself.  The  texts  (and 
examination)  are  available  to  both  members  and  nonmembers  of  NCMA.  For 
a  further  discussion  of  the  Handbook  and  Supplements,  the  reader  is 
referred  to  the  first  chapter. 

Endeavors  to  Improve.  NCMA's  connittee  on  the  Professional  Body 
of  Knowledge  is  responsible  for  maintaining  the  statement  that  defines 
the  Body  of  Knowledge.  According  to  NCMA,  the  statement  is  dynamic  and 
subject  to  change  in  order  to  keep  pace  with  the  maturing  and  changing 
profession  (7:x). 
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Figure  1.  NCMA  Body  of  Knowledge  (17:16) 
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Figure  1  (continued).  NONA  Body  of  Knowledge  (17:17) 
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NCMA  BODY  OF  KNOWLEDGE 
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NCMA  is  Bt4>portivs  of  rssssrch  sfforts  siasd  at  battering  ths 
fisld's  body  of  knowlsdgs.  Ovsr  ths  past  faw  yaars,  HCMA  has  aponsorad 
aa  affort  to  coapila  a  Bastar  listing  of  govanutent  acquisition  and 
contracting  tarns.  At  laaat  four  graduata  atudants  fron  tha  Naval 
Postgraduata  School  (M?S)  in  Montaray,  California,  and  fiva  graduata 
atudants  fron  tha  Air  Forca  Inatituta  of  Taehnology  (AFIT)  at  Wright- 
Pattaraon  Air  Forca  Basa,  Ohio  hava  foeusad  thair  ■aatara*  thaaaa 
toward  this  goal.  Tha  product  of  thair  rasaarch  will  ba  tha  avantual 
publication  of  a  rasaarch  basad  dictionary  of  taraa  by  NCMA. 

CPCM  Candidate* a  Elactronic  Workbook.  Ovar  tha  past  decada, 
innovations  hava  ehangad  tha  way  cowputars  atora,  Banaga,  and  retriava 
infomation.  According  to  Manion,  survival  akilla  in  the  use  of 
Bicrocoaputars  are  now  necessary  to  ba  succaaaful  in  our  changing 
society  (23:25).  In  education,  the  Bicrocowputer  has  bacoaw  an 
established,  well  recognized  node  of  instruction.  Rather  than  aak 
whether  tha  cowputer  should  ba  considered  for  use  is  instruction, 
teachers  ara  now  asking  how  particular  applications  can  ba  used  to  set 
up  a  curriculuB  (23:25).  Stated  again,  the  purpose  of  this  research  is 
to  develop  and  test  a  hypertext  version  of  tha  CPCM  Candidate's 
Workbook  and  Supoleawnts .  Hypertext  is  a  concept  Bade  possible  by 
innovations  in  cooputers  over  tha  past  decade. 

Sinply  put,  hypertext  docuaents  store  infornation  in  nodes 
connected  by  links.  Links  are  the  key  advantage  of  hypertext  docuaients 
over  conventional  paper  documents.  The  "linking  ability"  permits  a 
researcher  to  quickly  search  for  an  area  of  interest.  Once  a  desired 
section  is  located,  it  can  then  be  read  sequentially.  The  speed  that 
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tt*«rc  of  hypertext  systeas  can  browse  large,  volusiinous  docnaents  is 
its  chief  advantage  over  annual,  aore  cuabersoae  techniques.  This 
researcher  proposed  to  apply  hypertext  concepts  in  the  developaent  of 
an  electronic  workbook  that  can  provide  the  saae  inforaation  as  the 
text  version.  HsT^rtext  applications,  like  a  prototype  hypertext 
aedical  therapeutics  handbook,  have  already  proven  that  study  can 
bacoae  aore  than  a  passive  activity  of  viewing  linear  sequences  of 
straight  text  (11:68).  The  following  paragraphs  are  a  discussion  of 
key  teras,  types  of  hypertext  applications,  and  steps  required  to 
author  a  hypertext  application. 


Hypertext  History  and  Definition 

Detailed  descriptions  of  the  history  and  development  of  hypertext 

concepts  are  abundant  in  the  literature.  Schneidenoan  gave  the 

following  definition  of  hypertext  in  his  book  Hypertext  Hands  On ! : 

The  aost  cosnon  meaning  of  "hypertext”  is  a  database  that 
has  active  cross-references  and  allows  the  reader  to  "jump" 
to  other  parts  of  the  database  as  desired.  This  makes  the 
reading  (and  writing)  process  nonsequential.  It  is  the 
requirement  for  active  cross-references  that  sakes  a 
computer  necessary  to  iiiq>leaent  hypertext.  (32:3) 

Vannevar  Bush,  Science  Advisor  for  President  Roosevelt,  is  widely 

known  for  laying  the  groundwork  for  a  concept  known  as  hypertext  (6:20; 

31:168;  35:816).  Bush's  1945  article  in  Atlantic  Monthly  discussed  an 

innovative  machine  called  a  "aemex"  that  includes  on-line  text  and  is 

designed  to  allow  browsing  (35:816).  Although  Bush  never  built  the 

machine,  he  described  it  in  the  following  unner: 

A  device  which  an  individual  stores  his  books,  records,  and 
cosBxuii cat ions,  and  which  is  mechanized  so  that  it  may  be 
consulted  with  exceeding  speed  and  flexibility.  It  is  an 
enlarged  intimate  supplement  to  his  memory.  (35:816) 
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IWo  other  noted  hypertext  pioneers  are  Ted  Nelson  and  Douglas 
Bnglebart.  Bnglabart  proposed  the  systea  H-LAM/T  (Human  uaing 
Language,  Artifects,  and  Methodology,  in  which  he  is  trained)  in  1963 
(6:22).  Hia  attempt  was  the  iirst  to  build  the  neaez  described  by  Bush 
twenty  years  earlier  (35:816).  Ted  Nelson  coined  the  term  "hypertext" 
during  his  work  on  the  Xanadu  Project  in  1968  (6:23).  The  project's 
goal  was  to  hare  the  world's  knowledge  on-line  for  users  at  aultiple 
workatationa  (31:169). 

Interest  on  hypertext  has  grown  over  the  last  several  years. 
Although  no  one  reason  can  explain  the  expansion,  there  is  certainly 
agreement  on  the  role  that  technology  has  played.  Improved  coegmting 
power,  high-resolution  displays,  and  networking  are  mentioned  by 
several  authors  as  breakthroughs  that  have  contributed  to  the 
widespread  availability  of  hypertext  applications  (35:818;  24:21; 
32:ix-x).  Paul  Saffo,  in  the  article,  "What  You  Need  to  Know  About 
Hypertext,"  appropriately  reflected  on  the  tendency  of  technology  to 
catch  up  with  theory.  He  stated,  "If  we  are  in  the  midst  of  a 
revolution,  it  is  one  that  has  occurred  before  under  other  names" 
(31:173).  According  to  Saffo,  Owl  Systems  International  claims  that 
their  introduction  of  the  program  named  Guide  in  1985  was  the  first 
consercially  available  personal  congniter  hypertext  system  (31:171). 

Other  Definitions 

Variations  to  the  above  explanation  of  hypertext  exist  in  the 

literature.  Smith  and  Weiss  provided  one  iu  Conmunications  of  the  ACM. 

More  precisely,  hypertext  is  an  approach  to  information 
aanageaient  in  which  data  is  stored  in  a  network  of  nodes 
connected  by  links.  Nodes  can  contain  text,  graphics. 
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audio,  Tideo,  as  well  as  source  code  or  other  forms  of 
data.  The  nodes,  and  in  some  systems  the  network  itself, 
are  meant  to  be  viewed  through  an  interactive  browser  end 
manipulated  through  a  structure  editor.  (35:816) 

This  definition  is  replete  with  other  key  terms  that  deserve 

explanation.  They  are  nodes,  links,  browse,  and  network. 

Nodes.  Nodes  and  links  are  the  building  blocks  of  hypertext 

systems.  Schneidemsn  stated  that  a  node  is,  "The  smallest  unit  of 

information  in  a  hypertext  database  to  which  a  link  can  be  made" 

(32:159).  Nodes  should  express  one  idea;  however,  there  is  not 

consensus  on  their  recomnended  size  (32:6).  Node  length  can  vary  from 

an  entire  book  to  the  text  that  can  fit  on  a  single  screen  (25:12). 

In  her  thesis.  Use  of  Hypertext  for  the  Pevelonment  of  an  Office 
Reference  System  on  Economic  Analysis.  Captain  DeAnna  L.  McMurry 
provided  a  cong>rehensive  review  of  hypertext  literature.  Her 
discussion  of  nodes  made  the  distinction  between  article-*based  or  card-* 
based  nodes.  Artiele~baaed  nodes  .*epresent  single  documents  that  the 
user  can  scroll  through  one  screen  at  a  time,  while  card-based  nodes 
fit  the  text  on  a  single  screen.  Captain  McMurry  stated  that  users  of 
article-based  nodes  must  be  knowledgeable  of  scrolling  techniques  for 
the  system  to  be  effective.  Card-based  systems  eliminate  the 
associated  risk  of  inexperienced  users;  however,  text  must  be 
partitioned  to  infonution  that  can  fit  on  a  screen  display  (25:12-13). 

Hypermedia,  unlike  hypertext,  uses  ncdes  that  include  text, 
graphics,  digitired  speech,  audio  recordings,  pictures,  animation,  and 
film  clips  (5:18).  This  research  is  limited  to  nodes  that  are  text. 

In  addition,  this  discussion  of  hypertext  should  not  be  confused  with 
an  expert  system.  Although  expert  cystems  can  incorporate  hypertext 
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concepts,  an  expert  system  is  characterized  by  an  inference  engine  that 
can  make  decisions  based  on  new  infonsation  (16:846). 

Links.  Akscyn,  Conklin  and  Schneiderman  view  links  as  the  most 
important  concept  of  hypertext  systems  (1:829;  5:18;  32:3).  Links 
connect  nodes  (1:820),  and  can  be  thought  of  as,  "The  glue  that  holds 
hjrpertext  together"  (32:3).  Karen  Smith  further  defined  links  stating, 
"a  link  connects  words  or  sentences  in  one  electronic  document  to 
related  information  in  another  document"  (36:32).  When  a  link  is 
selected  (or  activated),  the  node  connected. to  it  is  retrieved  and 
quickly  displayed  on  the  screen  (16:837). 

The  designer  of  hypertext  software  can  use  a  number  of  available 
methods  to  mark  the  presence  of  links.  For  exas^le,  highlighted  or 
underlined  words  and  phrases  can  be  embedded  within  the  text  of  a 
document  (31:171;  20:312).  Visual  symbols  called  icons  can  also  be 
used,  and  appear  as  graphic  images  on  the  display  (32:67). 

Just  as  there  are  a  number  of  methods  to  embed  links,  there  are 
also  several  means  to  activate  a  link.  Use  of  cursor  keys,  mouse, 
track  ball,  or  touch  screen  are  four  possible  methods  (20:312). 

Since  links  connect  two  nodes  that  contain  information,  it  is 
reasonable  to  assume  that  users  might  desire  to  return  to  the  node  from 
idiich  they  originated.  The  ability  to  return  depends  on  whether  a  link 
is  unidirectional  or  bidirectional.  Schneiderman  defines  the  terms  in 
the  following  statements:  "A  unidirectional  link  goes  in  one  direction 
only;  a  bidirectional  link  goes  in  both  directions.  A  bidirectional 
link  allows  you  to  traverse  the  link  from  the  other  end"  (32:4). 
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Links  can  be  organized  using  a  strict  hierarchy,  or  network 
structures  (32:4,7).  Hierarchical  links  connect  information  that 
progressively  grow  in  detail  on  the  same  subject  (32:4).  Tree  diagrams 
can  be  used  to  document  hierarchical  links.  Nonhierarchical,  or 
network  links  connect  more  peripheral  siaterial,  such  as  comments  and 
cross-references  (1:828).  "Hierarchical"  and  "network”  (32:4)  are  used 
interchangeably  by  Akscyn  with  "tree  items"  and  "annotation  items" 
(1:828). 

Authoring.  Authoring  is  the  process  of  designing,  developing,  or 
creating  hypertext  systems. 

Browse .  To  browse  is  to  peruse  information  in  an  exploratory 
Btanner.  Raymond  and  Tompa  defined  browsing  as  a  two  stage  process. 
First,  a  subject  area  is  decided  on,  followed  by  quick  navigation  to 
its  location  (29:872). 

Graphical  Browsing  Systems.  Graphical  browsing  systems  are 
carefully  developed,  well-planned  maps  that  can  present  graphical 
overviews  of  the  network  structure  (16:840).  By  presenting  the  user 
with  a  "bird's-eye  view"  of  the  document  structure,  Karen  Smith  stated 
that  browsing  systems  can  answer  questions  like  "Where  am  I?",  "What 
information  is  available?",  and  "How  can  I  get  to  it?"  (36:39). 

Smith  reconnended  designers  of  hypertext  limit  maps  of  the  entire 
structure  to  smaller  networks  that  have  less  than  50  links  (36:40). 
Local  maps  are  store  appropriate  for  larger,  more  complex  documents. 

Application  Schemes 

Hypertext  applications  can  take  many  forms.  Some  authors 
categorize  educational  applications  as  either  a  tool,  tutor,  teacher. 


27 


or  tetter  (12:3-12).  Authors  of  literature  in  the  area  of  coiqputer- 
assitted  instruction  discuss  several  forms  that  applications  can  take. 
Kemner-Richardson ,  Lamos,  and  Rest  wrote  a  handbook  for  Air  Force 
Instructional  Managers.  In  it,  they  described  six  possible  forms  of 
computer-assisted  instruction:  1)  Informational;  2)  Drill-and-Practice; 
3)  Tutorial;  4)  Simulation;  5)  Inquiry;  and  6)  Intelligent  (19:19-24). 
In  their  research.  Captain  Richard  Fryer  and  Captain  Michael  Sivley 
found  the  above  six  categories  of  applications  as  the  most 
comprehensive  (12:3-13;  34:15). 

While  these  descriptions  of  computer-assisted  instruction  schemes 
are  valuable,  Captain  HcHurry's  review  of  hypertext  classifications  is 
much  more  apropos.  Captain  HcMurry  recognized  four  dimensions  of 
hypertext  systems  that  were  first  described  by  Nielson.  Shown  in  Table 
1,  this  versatile  classification  scheme  was  designed  to  identify  a 
hypertext  system  during  any  stage  of  development  (25:20-21). 

Captain  HcMurry  also  identified  a  method  used  by  Conklin. 

Conklin  combined  the  second  and  third  categories  of  Table  1,  and 
explained  the  concepts  in  greater  detail  to  arrive  at  the 
classifications  contained  in  Table  2  (25:20-21). 

In  Hypertext  Hands  On!.  Schneiderman  discusses  hypertext  schemes 
through  an  examination  of  how  they  have  been  applied.  Examples  include 
dictionaries,  encyclopedias,  medical  textbooks,  product  catalogs, 
creative  writing,  help  systems,  technical  documentation,  instruction, 
software  engineering,  religious  studies,  and  museum  exhibits  (32:20). 
This  research  does  not  replicate  any  of  the  above  hypertext  schemes. 
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TABLE  1 


Hypertext  Classification  Scheme  1  (25:21) 


generation 

original  (1945  to  1980)  and  cur¬ 
rent  (1980s) 

scope  of  the  user  target 

single  user,  work  group,  cor¬ 
porate  division,  and  the  whole 
world 

browsing  versus  authoring 

information  presentation  versus 
network  creation  and  manipulation 

task  specificity 

general,  some  task  specificity, 
and  task  specific 

TABLE  2 

Hypertext  Classification  Scheme  2  (25:21) 

macro  literary  systems 

.  .  .  large  on-line  libraries. 

..  [for]  all  publishing,  read¬ 
ing,  collaboration,  and  criti¬ 
cism  .  .  .  within  the  network. 

problem  exploration  tools 

tools  to  support  early  unstruc¬ 
tured  thinking  on  a  problem,  in 
which  many  disconnected  ideas 
come  to  mind  .  .  . 

browsing  systems 

similar  to  mac:  o  literary 
systems,  but  smaller  scale 
systems  for  teaching,  reference, 
and  public  information,  where 
ease  of  use  is  crucial. 

general  hypertext  technology 

general  purpose  systems  designed 
to  allow  experimentation  with  a 
range  of  hypertext  applications — 
most  conmonly  applied  to  reading, 
writing,  collaboration,  etc,.. 

Gagne  stated  that  the  classification  of  an  application  car  also 


be  based  on  the  "learning  outcome"  of  sessions  at  the  computer  (13:17). 
The  Workbook  is  published  to  assist  CPCM  candidates  organize  an 
approach  to  the  CPCM  exam,  and  to  provide  the  contents  of  past 
examinations  (7:xv).  The  learning  outcome  of  sessions  at  a 
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eooputerized  version  would  be  designed  for  similar  purposes.  This 
researcher  has  used  the  Workbook  for  informational,  research,  and 
reference  purposes.  Thus,  the  reference  system  proposed  by  this 
research  most  closely  resembles  a  browsing  system  categorized  within 
Table  2. 


System  Design 

There  is  an  abundance  of  literature  that  discusses  the  steps  that 
have  been  used  to  develop  particular  hjrpertext  applications.  The 
researcher  did  not  find  a  cookbook  approach  that  can  be  applied  to  the 
design  of  every  system.  However,  Ben  Schneiderman  provided  lists  of 
system  design  considerations  that  are  based  on  years  of  experience 
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re 


(33; 124-128;  32:62-63). 

Medical  Handbook  and  Oxford  English  Dictionary.  In  the  article 
"Searching  for  Information  in  a  Hypertext  Medical  Handbook,”  Mark 
Frisse  recounted  four  steps  that  he  and  others  used  to  convert  a 
Bwdical  textbook  into  a  hypertext  oanual.  The  manual  was  first 
partitioned  into  individual  cards  based  on  the  inherent  hierarchies 
structure  of  the  text.  Second,  the  first  six  words  of  sections  wee 
assigned  a  card  title.  Structural  links  between  each  card  and  its 
parent  were  created  third.  The  design  of  indexing  and  retrieval  me^ 
finalized  the  process  (11:881). 

In  their  article,  "Hypertext  and  the  Oxford  English  Dictionat 
Raymond  and  Tompa  described  system  design  as  an  overall  problem  of 
defining  text  fragments  and  the  associated  links  (29:871).  It  appears 
that  these  authors  combined  Frisse 's  steps  into  one  overall  task. 
Raymond  and  Tompa  also  recomnended  careful  examination  of  the 
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ry. 


30 


conventional  document's  structure  and  consultation  with  experienced 
professionals  if  possible  (29:871). 

System  Design  Steps.  Hypertext  is  a  new  concept,  and  every 
project  is  unique.  Designers  apply  their  experience  and  training  while 
authoring  systems,  and  final  products  are  representative  of  each 
author's  style.  Although  the  process  can  be  considered  both  an  art  and 
a. craft,  Ben  Schneiderman  provided  a  comprehensive  list  of 
considerations  in  Hypertext  Hands  On!  that  deserve  mentioning  here. 

1.  Chunking  -  The  information  to  be  presented  needs  to  be 
organized  into  small  "chunks"  that  deal  with  one  topic, 
theme,  or  idea.  Each  chunk  represents  a  node  or  document 
in  the  database. 

2.  Interrelationships  -  Each  document  should  contain  links 
to  other  documents.  The  more  links  contained  in  the 
documents,  the  richer  the  connectivity  of  the  hypertext 
database.  On  the  other  hand,  avoid  gratuitous  links  each 
link  should  serve  a  clear  purpose. 

3.  Consistency  of  document  names  -  It  is  important  to  keep 
a  list  of  names  given  to  documents  as  they  are  created; 
otherwise,  it  becomes  difficult  to  identify  links  properly. 
Synonyms  are  to  be  encouraged  (as  long  as  they  are  kept 
track  of). 

4.  Master  reference  list  -  Create  a  master  reference  list 
as  you  author  to  ensure  correct  citations  and  prevent 
redundant  or  missing  citations. 

5.  Sinqplicity  in  traversal  -  Navigation  should  be  simple, 
intuitive,  and  consistent  throughout  the  system.  Movement 
through  the  system  should  be  effortless  and  should  not 
require  thought . 

6.  Screen  design  -  Screens  should  be  designed  so  they  can 
be” grasped  perceptually.  Visual  layout  is  very  inq>ortant 
in  screen  design. 

7.  Low  cognitive  load  -  Minimize  the  burden  on  the  user's 
shortterm  memory.  Do  not  require  the  user  to  remember 
things  from  one  screen  to  another. 

8.  Early  reviews  -  Subject  the  database  to  technical, 
legal,  and  management  reviews  as  early  as  possible.  As  the 
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database  bacones  larger,  changes  become  more  difficult  to 

make. 

9.  Maintain  sultiple  perspectives  -  When  authoring,  try  to 
balance  the  technical  requirements  of  the  system  with  the 
user's  perspective  and  the  organizational  use  of  the 
database.  (32:62-63) 

Authoring  hypertext  systems  is  an  iterative  process  that  might 
involve  all  or  some  of  the  above  steps.  In  addition  to  these 
strategies,  there  are  a  number  of  in^rtant  issues  that  should  be 
considered  during  the  overall  design  process. 

Ease  of  Use.  The  technology  that  was  the  genesis  of  interest  in 
hypertext  systems  also  presents  dangers  that  can  limit  their 
effectiveness.  .  High  resolution  displays,  enormous  processing  power, 
networks,  the  ability  to  link  across  large  databases,  and  audio/video 
sequences  are  a  few  attributes  of  hypertext  technology.  If  used 
improperly,  the  result  could  be  poorly  designed  products  that  scare 
people  away  (15:38).  'ossett  directly  addressed  faddish  additions  like 
graphics  or  video  to  instructional  programs  that  add  considerable 
expense.  He  recommended  that  a  rational  examination  of  these 
expensive,  technological  improvements  should  be  made  first  to  determine 
if  the  gains  can  justify  the  costs  (10:43). 

In  light  of  the  above  discussion,  a  number  of  authors  recommend 
ease  of  use  as  a  primary  goal  for  any  hypertext  system  (1:834;  5:38; 
10:43;  15:38;  20:314;  35:22).  The  linking  ability  that  makes  hypertext 
tuiique  also  presents  an  inherent  danger.  The  power  that  allows  quick 
navigation  throtigh  a  document  can  confuse  users  to  the  point  where  they 
are  "lost  in  space"  (15:38;  32*49).  The  disorientation  described  here 
is  usually  due  to  a  lack  of  sufficient  information  of  a  user's  current 
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location  relative  to  the  overall  structure  of  the  document  (32:49). 
Raymond  recommends  careful  planning  of  the  document's  structure  during 
the  early  design  stages  to  improve  user  orientation  (29:871).  There 
are  also  a  number  of  mechanisms  that  can  be  used  for  the  same  purpose. 
Graphical  browsers,  indexes,  bidirectional  links,  and  bookmarks  are  a 
few  examples  of  tools  that  are  available  to  the  author.  Bookmarks  are 
electronic  markers  that  allow  a  user  to  return  directly  to  a  siarked 
location  in  a  system  (32:14). 

User  frustration  can  also  be  avoided  by  designing  a  system  that 
responds  quickly.  Akscyn  reconmended  fast  link  selection  response 
times  (of  less  than  .25  seconds)  to  best  reduce  mental  fatigue,  time, 
and  physical  effort  (1:828).  He  stated,  "He  believe  the  ability  to 
browse  quickly  in  a  hypermedia  system  is  critical  to  its  usability" 
(1:830).  A  caveat  here  is  to  beware  of  making  link  response  too  fast, 
otherwise  novices  will  not  recognize  screen  changes  (32:39). 

Simmarv 

This  chapter  first  discussed. professionalism  in  contracting  from 
a  historical  perspective.  Because  of  laws  like  the  Defense  Acquisition 
Workforce  Improvement  Act,  more  and  more  people  will  be  pursuing 
continued  education  in  contracting,  and  professional  recognition 
through  programs  like  the  Certified  Professional  Contracts  Manager. 

Hypertext  was  the  second  topic  discussed  in  this  chapter.  The 
history,  pertinent  definitions,  application  schemes,  and  system  design 
were  all  addressed.  Hypertext  authoring  tools  offer  great  versatility. 
However,  experienced  authors  recomH^nd  rational  decisions  be  made  in 
order  to  avoid  unnecessary  cost,  time,  and  con^lexity. 
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Although  there  are  no  standard  steps  to  follow  to  create  a 
hypertext  document ,  general  guidelines  are  available.  It  is  up  to  the 
individual  to  tailor  a  system  to  the  intended  user  while  keeping 
overall  design  issues  in  mind. 
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Introduction 

This  chapter  begins  by  first  describing  the  general  method 
employed  to  solve  the  research  problem  and  the  rationale  for  its  use. 
The  five  objectives  introduced  in  Chapter  I  are  then  addressed.  Once 
again,  the  objectives  were:  1)  Obtain  permission  to  create  a  hypertext 
version  of  NCMA's  CP CM  Candidate *s  Workbook  and  Supplements :  2)  Select 
and  purchase  the  software;  3)  Develop  the  hypertext  model;  4)  Test  and 
evaluate  the  hypertext  system;  and  5)  Inclement  feasible  improvements. 
The  objectives  are  discussed  in  light  of  the  methodology  used  to 
accon^lish  the  specific  research  problem. 

Research  Method 

Prototyping  was  the  general  method  used  to  structure  the  software 
development  process.  Prototyping  was  chosen  since  it  is  a  method  that 
researchers  can  use  to  quickly  determine  the  feasibility  of  a  model, 
and  then  test  its  usefullness.  Objectives  two  through  five  embrace  the 
prototyping  effort.  Objective  one  began  the  research  because  the  other 
objectives  were  contingent  upon  obtaining  copyright  permission.  The 
paragraphs  below  will  explain  the  requirement  for  obtaining  copyright 
permission  in  more  detail. 

The  general  method  employed  to  solve  the  specific  problem  was 
modeled  after  the  procedures  used  by  Captain  HcMurry,  and  was  tailored 
to  meet  the  needs  of  this  research  (25).  For  a  detailed  discussion  of 
prototyping  and  life-cycle  models,  the  reader  is  inferred  to  her  thesis 
(25:38-54). 
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gbitctiYft  It _ Ohfciin,..Gapyright ..Pierffiigjipn 

Tha  potential  o£  hypertext  applications  has  been  described  as 
iaaeasiirable .  However,  the  individuals  who  have  theorized  global 
libraries  of  text  have  also  introduced  copyright  law  issues  and 
questions.  According  to  Schneiderman,  "Copyright  law  is  designed  to 
protect  the  intellectual  rights  of  an  author  on  original  works  and  all 
derivations"  (32:53). 

Henry  Jones,  who  wrote  "Developing  and  Distributing  Hypertext 

Tools:  Legal  Inputs  and  Parameters,"  discussed  the  five  rights 

protected  by  copyright  law.  According  to  Jones, 

The  iBgMrtanee  of  copyright  law  to  the  development  of 
hypertext  tools  is  underscored  by  the  fact  that  this 
portion  of  federal  law  regulates  not  merely  the  right  to 
reproduce  a  work  (e.g.  to  "copy").  In  fact,  a  copyright  is 
a  bundle  of  five  distinct,  severable  rights.  The  other 
four  are  the  right  to  adapt  or  modify  a  work  (i.e.,  to 
create  a  "derivative  work"),  and  the  rights  to  publicly 
distribute  copies  of  a  work,  to  display  it  (e.g.,  on  a 
computer  terminal),  or  to  perform  it.  (18:368) 

Jones  summed  his  article  up  with  the  following  advice: 

The  moral:  if  you  plan  to  reproduce,  modify  or  display  the 
code,  proprietary  structure  and  design,  or  documentation  of 
another  developer,  you  may  need  a  written  license 
agreement.  In  any  event,  early  discussion  and  coordination 
with  copyright  counsel,  based  on  a  specific  review  of  both 
your  planned  activities  and  the  materials  originally 
created  by  third  parties,  can  help  identify  your 
obligations  and  rights.  (18:370) 

This  research  was  limited  to  the  conversion  of  one  document  (and 

its  suppleawnts)  into  hypertext  format.  The  following  statement  is 

found  on  the  first  page  of  the  CPCM  Candidate's  Workbook: 

Copyright  1987.  All  rights  reserved.  Reproduction  in  any 
form  expressly  prohibited.  No  part  of  this  publication  may 
be  reproduced  or  distributed  in  any  form  or  by  any  means, 
or  stored  in  a  database  or  retrieval  system,  without  the 
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prior  writtan  parmiBsion  of  tha  National  Exacutive  Diractor 
of  NCMA.  (7:i) 

Thua,  tha  attainment  of  parmisaion  from  NCMA  waa  a  vital  firat  atep. 


Obiactiva  2; 


Select  and  Purchase  Hypertext  Software 


Tha  aecond  objective  waa  to  identify  the  important 
eharacteriatiea  of  hypertext  aoftware  needed  for  thia  research.  The 
ehoieaa  ware  limited  to  thoaa  that  run  on  tha  MS-DOS  operating  aystem 
software.  NCMA  Headquarters  uses  this  operating  system,  as  does  the 


Air  Force  Institute  of  Technology. 

Since  links  and  nodes  are  the  key  attributes  of  hypertext, 
authoring  systems  should  amnage  these  two  issues  effectively  (33:121). 
Indexes  of  nodes  and  links,  in  addition  to  the  efficient  manipulation 
of  these  items  are  also  of  great  value  to  a  hypertext  authoring  system 
(33:121).  In  addition  to  these  features  are  several  others: 

-  range  of  editing  functions  available  (i.e.,  copying, 
moving,  insertion,  deletion,  global  change  within  an 
article,  etc.). 

-  the  availability  of  lists  of  link  names,  index  terms, 
synonyms,  etc. 

-  the  range  of  display  formatting  commands  available. 

-  the  availability  of  search/replace  functions  for  making 
global  changes  across  multiple  nodes. 

■*  control  of  color  (text, background) :  color  can  make  the 
text  look  attractive,  but  it  can  also  be  distracting. 

Since  users  are  very  different  in  their  preferences  and 
tasks,  it  should  also  be  possible  for  users  to  reset  color 

usage  parameters. 

-  the  capability  to  easily  switch  between  author  and 
browser  modes  to  test  ideas. 


-  accessing  CD-ROM,  videodisc,  or  other  devices:  new 
devices  are  emerging  regularly  with  remarkable  storage 
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capabilities.  It  should  be  possible  to  access  information 
on  a  variety  of  devices. 

-  capability  to  export  files  to  other  systems. 

■*  operability  on  a  local  network. 

-  multi-user,  network  &  distributed  databases. 

-  version  keeping  (can  old  versions  of  an  article  be 
stored?). 

-  graphics  &  video  facilities:  are  there  esibedded  graphics 
editors  and  swchanisms  for  exploring  the  videodisc. 

-  collaboration:  can  BX>re  than  one  person  edit  the  database 
at  one  time?  Can  coagMnents  of  the  database  written  by 
several  people  be  merged  into  one  hypertext  database. 

-  data  eom^ression:  compression  algorithms  can  reduce  the 
size  of  the  database  and  facilitate  distribution  of  disks 
or  disseiuination  by  electronic  networks. 

-  security  control:  is  there  password  control  for  the 
database  or  parts  of  it. 

-  encryption:  can  sensitive  nodes  be  encrypted? 

-  reliability:  does  the  software  perform  without  bugs,  and 
without  losing  data? 

-  integration  with  other  software/hardware. 

-  browser  distribution:  does  every  user  of  the  hypertext 
have  to  acquire  a  copy  of  the  full  system  or  can  the 
browsing  part  be  included  with  the  database.  (18:121-124) 

The  above  criteria  were  considered  by  Captain  McMurry  during  her 

research.  The  following  eight  were  considered  important  by  Captain 

McMurry  for  the  development  of  her  prototype:  index  of  nodes,  easy 

management  of  links,  wide  range  of  editing  functions,  capability  of 

easy  switching  between  editor  and  browser,  version  keeping,  data 

compression,  relisbility,  and  browser  distribution  (25:44).  Similarly, 

several  of  the  features  were  given  special  interest  by  this  researcher. 
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Th«7  included  easy  numagement  of  nodes  and  links,  editing  functions, 
data  compression,  integration,  and  reliability. 

Three  other  issues  were  considered  before  the  final  selection  of 
a  hypertext  system.  The  first  was  the  need  for  a  royalty-free  runtime. 
Runtime  programs  are  a  limited  version  of  the  cos^lete  hypertext  system 
software.  Large  numbers  of  users  can  use  runtime  programs  without 
having  to  purchase  the  entire  authoring  software.  The  second  issue  was 
the  need  for  a  familiar  editor  or  word  processor  style  interface.  This 
was  due  to  the  fact  that  the  researcher  lacked  programming  skills.  In 
addition,  time  constraints  demanded  a  short  learning  curve  for  becoming 
proficient  in  the  use  of  the  authoring  system. 


Obiective  3;  Develop  the  Hypertext  Model 

The  topic  of  system  development  was  researched  in  the  hypertext 
literature.  In  addition  to  the  system  design  considerations  listed  in 
Chapter  II,  the  following  are  seven  more  recomnended  by  Ben 
Schneiderman: 


1.  Build  the  project  around  the  structuring  and 
presentation  of  information,  not  around  the  technology. 
Make  sure  that  the  structure  matches  users '  needs . 

2.  Develop  a  high  concept  of  clear  (simple,  clear,  bold 
idea)  for  the  body  of  information  you  are  organizing. 
Avoid  fuzzy  thinking  when  creating  the  information 
structure . 


3.  Keep  the  navigational  structure  simple  and  intuitive. 
Find  a  simple,  comprehensive,  and  global  structure  chat  the 
user  can  easily  develop  into  a  cognitive  map. 

4.  Bach  screen  should  be  self-contained  and  should  not 
require  the  user  to  search  his  or  her  memory  to  understand 
it  or  navigate  from  it.  Be  sensitive  to  the  possibility 
that  the  user  will  get  "lost  in  hyperspace,”  and  develop 
the  system  so  recovery  is  sinq>le. 


39 


5.  Keep  the  user's  effort  very  low.  Reslize  that  the  user 
is  allocating  attention  among  multiple  tasks:  navigation, 
processing  the  content  of  the  database,  and  attending  to 
the  business  at  hand. 

6.  Users  are  your  best  source  of  feedback;  use  them 
throughout  the  development  process  to  test  your  designs. 

Realize  that  you  are  not  a  good  judge  of  your  own  design 
because  you  know  too  much. 

7.  Study  the  target  population  of  users  carefully  to  make 
certain  you  know  how  the  system  will  really  be  used. 

Create  demonstrations  and  prototypes  early  in  the  project; 
don't  wait  for  the  full  technology  to  be  ready.  (32:124) 

Broad  objectives  like  those  found  above  are  targeted  for  use 

throughout  the  development  process.  They  do  not  provide  a  systematic, 

sequential  approach  for  the  inexperienced  author.  Review  of  the 

literature  revealed  a  lack  of  step-by-step  approaches  for  development 

of  hypertext  documents.  There  are  two  reasons  for  this.  First, 

hypertext  itself  is  a  developing  technology.  Second,  each  application 

is  unique,  and  a  standard  approach  for  each  is  sioqply  impossible. 

Three  steps  were  used  to  structure  the  software  development 

effort.  The  first  step  was  to  design  the  software  structure  on  paper. 

Second,  the  text  was  converted  to  machine  readable  format.  Third, 

methods  were  developed  to  present  the  information  to  the  user. 

Obiective  3a  -  Structure.  The  introductory  screen  of  a  hypertext 

system  often  displays  the  document's  structure.  Preplanning  the 

document  structure  can  lead  to  a  better  designed  access  tool.  Upon 

"v. 

5 

entering  a  hyperc-Mt  program,  users  must  bo  able  to  decide  where  to 
branch  to  get  information  of  interest.  This  activity  should  be  able  to 
be  performed  with  a  minimum  amount  of  confusion.  Hypertext  structure 
is  i]q)licit;  the  system  should  clearly  define  the  structure  for  the 
user.  An  explicitly  designed  approach  can  make  a  system  useful  to  a 
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n«w  user  in  a  niatter  of  minutes,  while  repeated  use  of  the  system  helps 
reinforce  the  structure. 

The  need  for  preplanning  the  document  structure  also  stems  from  a 
practical  perspective.  Changing  the  link  structure  of  a  document  well 
into  the  development  phase  can  result  in  frustration  and  an  increase  in 
development  time.  Captain  McMurry  reconsidered  the  link  structure 
after  ingwrting  ASCII  files  while  creating  an  office  reference  system 
on  economic  analysis.  She  reflected  that  substantial  time  could  have 
been  saved  if  more  time  had  been  spent  preplanning  before  importing  the 
text  (25:58). 

The  introductory  screen  discloses  a  document's  structure,  and  is 
a  key  design  issue.  One  author  condensed  several  approaches  that  have 
been  used  into  three  separate  strategies. 

1.  Make  the  root  document  an  overview  that  contains  links 
to  all  major  concepts  in  the  database  [glossary  strategy]. 

2.  Adopt  a  hierarchical  approach  in  which  the  links  in  the 
root  docximent  are  major  categories  [top-down  strategy]. 

3.  Organize  the  root  article  as  a  list  or  table  of 
contents  of  the  major  concepts  in  the  database  [menu 
strategy].  (32:126) 

The  most  appropriate  strategy  is  determined  by  the  purpose  and  use  of 
the  hypertext  document. 

The  Workbook  and  Supplements  which  were  converted  to  hypertext 
are  well  structured  documents.  In  addition  to  the  introductory 
material,  the  Workbook  is  comprised  of  nine  categories.  The  categories 
are:  required,  general,  legal,  financial,  production,  contracting, 
logistics,  conmercial,  and  state/local.  The  hierarchical  approach 
appeared  to  be  best  suited  for  this  application.  The  actual  document 
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resembles  a  hierarchy  of  past  exam  questions,  and  a  replicated 
hypertext  version  would  be  familiar  to  those  users  who  have  used  the 
Workbook. 

Objective  3b  -  Text  Conversion.  Again,  the  literature  regards 
the  conversion  of  text  into  stachine  readable  format  to  be  a  major 
concern  for  hypertext  developers  (32:127).  Text  conversion  has  become 
more  feasible  through  the  use  of  microcomputers,  and  standard  ASCII 
files.  Hypertext  authoring  systems  that  can  import  ASCII  text  files 
provide  for  efficient  development,  because  the  step  of  link 
identification  can  follow  much  more  quickly. 

Converting  large  documents  into  ASCII  files  requires  the  use  of  a 
acanner.  In  order  to  cvoid  monumental  editing  tasks,  past  hypertext 
developers  recomnend  investment  in  the  best  quality  scanning  equipment 
possible  (25:58). 

Objective  3c  -  Method  of  Presentation.  The  third  step  was  to 
devellop  a  method  to  present  the  information  to  the  user.  There  were 
three  major  decisions  that  had  to  be  made  in  order  to  meet  this 
objective.  The  first  was  to  determine  whether  to  use  an  article-based 
or  card-based  metaphor.  The  second  decision  was  to  decide  on  which 
combination  of  organizational  and  content  nodes  to  use.  The  final 
decision  was  to  ascertain  which  coomands  to  include  in  a  control  panel 
that  was  included  at  the  bottom  of  every  screen.  Each  of  these  areas 
are  qiscussed  in  greater  detail  below. 

Article-Based  or  Card-Based  Metaphor.  Article-based  and 
card-jbased  metaphors  are  the  two  primary  methods  used  to  present 
infoiUation  to  users  of  hypertext. 
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"Article  systems  utilize  the  metaphor  of  presenting  information 
to  the  users  as  an  article  -  continuous  text  that  scrolls  onscreen" 
(28:5).  Scrolling  is  a  technique  that  is  used  to  view  documents  that 
are  converted  to  article-based  metaphors.  The  page-up  or  page-down  key 
is  used  to  view  the  text,  such  like  a  person  turns  the  pages  of  a  book. 
This  type  of  system  is  only  effective  if  users  think  to  scroll  down  for 
further  information  (28:5).  Otherwise,  the  information  is  considered 
inaccessible. 

The  second  type  of  metaphor  is  a  card-based  system  (28:5).  Card- 
based  systems  present  information  through  the  use  of  an  index  card 
metaphor.  Scrolling  problems  are  eliminated  since  each  window  is 
limited  to  the  information  that  can  fit  on  a  single  screen. 

Organizational  or  Content  Nodes.  The  second  decision  was 
to  determine  which  combination  of  organizational  and  content  nodes  to 
use. 

Organizational  nodes  are  used  as  reference  points  for  users. 

They  can  contain  information  such  as  an  index,  table  of  contents, 
table,  or  figure  listings.  These  types  of  metaphors  are  valuable  in 
linear  texts,  and  shouldn't  be  abandoned  when  creating  hypertext 
(28:21-22). 

Content  nodes  contain  the  information  of  the  linear  doctiment . 
Links  located  in  an  organizational  node  would  be  connected  to  content 
nodes  where  the  substance  of  a  document  is  located  (28:22). 

Control  Panels.  The  final  decision  was  to  determine  which 
conraands  to  include  in  a  control  panel.  A  control  panel  is  a  useful 
tool  that  is  located  at  the  bottom  of  every  screen,  and  includes 
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ccnanonly  used  navigational  comnands  (28:155).  A  few  of  the  choices 
inside  a  control  panel  could  be:  Cover  Page,  Index,  Next,  Previous,  or 
Exit.  Users  depend  on  the  comnands  to  perforru  routine  tasks  without 
having  to  memorize  comnands  or  return  to  an  organizational  node. 
Control  panels  take  up  to  four  to  five  lines  of  screen  space;  however, 
they  are  considered  valuable  tools  that  outweigh  the  loss  of  a  fe<r 
lines  (28:156). 


Evaluation 


The  fourth  objective  was  to  receive  feedback  on  the  hypertext 
system.  A  questionnaire  was  usrd  as  the  primary  method  of  collecting 
data,  and  is  shown  in  Appendix  A. 


The  questionnaire  was  muddled  after  one  develoc-i^d  by  Captain 


McMurry  who  used  the  questionnaire  to  test  the  macro  level  of 
acceptance  of  the  system  (25:90'‘92).  Testing  for  macro  acceptance  was 
appropriate  in  order  to  assess  the  initial  success  of  the  design 
characteristics  (25:52-53).  The  questions  were  structured  to  measure 
"  .  .  .  ease  of  learning,  low  error  rates,  and  subjective  satisfaction 
(25:53). 


Ease  of  learning  refers  to  the  time  required  to  learn  the 
comnands  to  use  the  system;  error  rate  refers  to  the  errors 
made  in  carrying  out  a  specified  set  of  tasks;  and 
subjective  satisfaction  refers  to  how  much  users  liked  the 
overall  aspects  of  the  system.  (25:53) 


Volunteers  were  solicited  from  the  Air  Force  Institute  of 


Technology,  and  were  given  the  opportunity  to  complete  the 
questionnaire  at  their  personal  residence.  Time  constraints  were  not 
given  for  evaluation  of  the  model  or  completion  of  the  questionnaire. 


Since  the  population  included  individuals  with  contracting 
backgrounds,  the  instrument  also  included  questions  that  measured  the 
desire  to  possibly  purchase  the  program  at  some  point  in  the  future 
from  NCMA. 

Obiective  5t  Implement  Feasible  Improvements 

The  final  objective  of  the  research  was  to  implement  improvements 
that  were  identified  during  the  testing  and  evaluation  phase. 

Summary 

Chapter  III  described  the  objectives  that  guided  the  research 
effort.  The  first  objective  was  to  obtain  permission  from  NCMA  to 
aut«nate  the  CPCM  Candidate's  Workbook.  The  second  objective  was  to 
identify  and  purchase  hypertext  software  that  met  important  criteria. 
The  third  objective  was  to  develop  the  hypertext  software  once  the 
authoring  tool  was  purchased.  This  involved  three  steps:  1)  design  of 
the  structure;  2)  conversion  of  the  text  into  machine,  readable  format; 
and  3)  development  of  a  method  of  presentation.  The  fourth  objective 
was  to  test  and  evaluate  the  hypertext  model.  The  final  objective  was 
to  improve  the  system  based  on  feedback  from  the  questionnaires. 


IV.  Results 


While  Chapter  III  described  the  five  objectives  that  guided  the 
research,  this  chapter  reports  how  the  five  objectives  were  actually 
performed  during  the  development  process. 

Obiective  1:  Obtain  Copyright  Permission 

The  receipt  of  copyright  permission  was  the  critical  first  step 
of  the  methodology  in  order  to  comply  with  copyright  law  and  to  protect 
the  rights  of  the  original  author  of  the  Workbook  and  Supplements . 
Permission  was  granted  in  writing  by  the  Executive  Director  of  the 
National  Contract  Management  Association  to  reproduce  the  Workbook  for 
the  purpose  of  creating  a  computerized  version  (14).  A  copy  of  the 
letter  is  located  at  Appendix  B.  In  addition  to  granting  copyright 
permission,  the  letter  reveals  the  hopes  of  NCMA  to  receive  a  finished 
product  that  would  be  worthy  of  contributing  to  NCMA's  Educational 
Product  offerings  (14). 

Obiective  2;  Select  and  Purchase  Hypertext  Software 

HyperWriter  (TM)  was  the  authoring  system,  used  to  support  this 
research.  The  application  was  chosen  based  on  a  number  of  criteria 
that  were  discussed  in  the  previous  chapter.  In  addition,  the  previous 
research  of  Captain  McMurry  was  considered  during  the  selection 
process. 

Captain  McMurry  compiled  a  list  of  hypertext  systems  along  with 
the  address  and  telephone  numbers  of  the  companies  that  sell  them 
(25:89).  Each  system  was  reviewed  against  the  important  criteria  that 
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are  listed  in  Chapter  III.  HyperHriter  was  chosen  by  this  researcher 
becaxise  it  best  fulfilled  the  following  criteria: 

1)  HyperWriter  runs  on  MSDOS. 

2)  HyperHriter  offers  a  royalty^free  runtime. 

3)  HyperHriter  uses  a  familiar  word  processor  style  interface 
that  precludes  the  use  of  con^lex  programming  skills. 

4)  The  use  HyperHriter  is  simple  enough  to  learn  without  formal 
training  in  order  to  author  hsrpertext  systems  in  a  limited  amount  of 
time. 

Captain  McMurry  considered  the  first,  third,  and  fourth  criteria 
to  be  important  during  her  selection  process,  and  selected  HyperHriter 
for  her  research  (25:55). 

The  manufacturer  of  HyperHriter  (NTERGAID)  was  contacted  prior  to 
the  purchase  of  the  software  to  discuss  the  use  of  the  authoring  system 
for  the  specific  purpose  of  this  research.  The  staff  at  NTERGAID  felt 
confident  that  HyperHriter  could  be  used  to  develop  a  h3rpertext  version 
of  the  Horkbook .  In  addition,  a  HyperHriter  demonstration  diskette  was 
reviewed  to  get  a  "hands-on"  fueling  for  the  system.  All  of  the  above 
factors  contributed  to  the  selection  of  HyperHriter  for  this  research. 

Objective  3:  Develop  the  Hypertext  Model 

1 

The  researcher  originall^  planned  to  develop  the  hypertext  model 

by  following  the  objectives  frpm  Chapter  3  i  sequence.  The  following 

! 

were  the  steps:  1}  determine  t)ie  structure;  2)  convert  the  text  to 
standard  ASCII  files;  and  3)  to  develop  the  method  of  presentation. 

i 

All  three  of  the  steps  were  performed  during  the  development  of  the 
actual  hypertext  system,  but  not  necessarily  in  that  order.  For 
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•xaaple,  several  iterations  of  steps  one  and  three  were  performed, 
while  the  conversion  of  text  to  standard  ASCII  files  (step  2)  was 
accoBy>lished  only  once. 

As  mentioned  earlier,  the  Workbook  consists  of  nine  areas  of 
concentration  that  each  contain  past  examination  questions  presented  in 
a  standard  format.  In  order  to  save  time,  an  initial  system  was 
developed  that  contained  only  one  area  of  concentration.  Once  an 
initial  siodel  of  the  system  was  conqtleted,  the  development  process  was 
repeated  for  the  other  eight  areas  of  concentration.  Macros  were  used 
wherever  possible  to  minimize  the  time  required  to  replicate  the 
original  files,  and  to  standardize  the  file  format  (spaces,  window 
margins,  text,  etc.). 

The  following  paragraphs  discuss  the  three  steps  of  the 


development  process  in  detail. 


Obiective  3a  -  Structure.  After  HyperWriter  was  purchased, 
approximately  twelve  to  fifteen  hours  were  spent  reading  the  manuals 
and  getting  familiar  with  the  features  of  the  authoring  system.  Once  a 
basic  understanding  of  HyperWriter  and  its  abilities  were  achieved,  the 
researcher  designed  the  document’s  structure  on  paper.  The  initial 
drawing  served  as  a  starting  point  to  allow  the  other  steps  to 


progress.  Several  changes  were  made  to  the  structure  during  the 
development  process,  and  Appendix  C  illustrates  the  final  layout  of  the 


system. 

The  presentation  of  information  must  meet  a  user's  needs,  and  a 
system's  value  is  dependent  upon  a  navigational  structure  that  is  clear 
and  intuitive  (32:124).  Thus,  the  map  of  the  system  proved  to  be  an 
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•ssential  clement  of  the  research.  The  researcher  tested  numerous 
designs  on  the  user  before  a  final  layout  was  achieved.  Meaningful 
changes  were  even  made  during  the  latter  part  of  the  development 
process.  The  changes  to  the  structure  made  a  significant  contribution 
toward  the  development  of  a  cognitive  map.  The  latter  changes  were 
based  on  the  feedback  of  an  individual  who  was  not  previously  familiar 
with  the  research.  The  following  statement  made  by  Schneiderman  was 
definitely  worth  consideration:  "Realize  that  you  are  not  a  good  judge 
of  your  own  design  because  you  know  too  much"  (31:124). 

Obiective  3b  ~  Text  Conversion.  Conversion  of  the  text  to 
standard  ASCII  files  was  a  repetitive  and  time-consuming.  The 
conversion  process  actually  took  two  separate  phases.  First,  the 
Workbook  and  Supplements  were  scanned  one  page  at  a  time.  Second,  the 
text  was  imported  into  a  text  editor  to  remove  spelling  errors. 

The  entire  Workbook  and  Supplements  were  scanned  over  a  matter  of 
17  hours.  The  editing  process  was  prolonged  due  to  errors  introduced 
by  the  scanner  used  by  the  researcher.  For  example,  the  scanner  read 
one's  as  lower-case  L's,  zero's  as  O's,  and  it  included  random 
characters  such  as  \?  throughout  the  text.  Editing  of  the  text  took 
approximately  50  hours  total . 

Obiective  3c  -  Method  of  Presentation.  Every  hypertext 
application  presents  information  to  the  user  differently.  Three 
decisions  had  a  major  impact  on  the  particular  method  of  presentation 
used  by  the  final  hypertext  system.  Determinations  were  made  regarding 
the  t3rpe  of  metaphor  to  use  (card-based  or  article-based),  the 
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eoabination  of  nodes  to  use  (orgsnizstional  and  content),  and  the 
eosaands  to  include  in  a  control  panel  at  the  bottom  of  every  screen. 

Early  attesq;>ts  to  design  the  initial  document  involved 
experimentation  with  all  three  siajor  decision  areas.  It  was  in^ssible 
to  determine  one  method  of  presentation  without  effecting  the  others. 
The  docuaient  took  nearly  95  hours  to  develop.  Approximately  two^thirds 
of  the  total  time  was  used  to  design  the  initial  system,  while  the 
remsinder  involved  the  creation  of  the  other  eight  areas  of 
concentration.  Each  of  the  three  decision  areas  are  discussed  in 
detail  below. 

Article-Based  or  Card-Based  Metaphor.  The  decision  to  use 
an  article-based  or  card-based  metaphor  was  critical.  Once  the  text 
was  converted  to  ASCII  files,  a  determination  had  to  made  regarding  how 
the  questions  would  be  broken  down.  A  card-based  metaphor  is  much  more 
difficult  to  design  than  an  article-based  one  because  all  of  the 
information  must  be  broken  down  into  chunks  of  text  that  fit  on  a 
single  screen  (28:20). 

The  researcher  decided  to  use  an  article-based  metaphor.  The 
decision  was  based  primarily  on  the  size  of  the  questions  (number  of 
lines)  contained  in  the  Workbook .  and  the  availability  of  a  feature 
offered  by  HyperWriter.  A  great  number  of  the  CPCM  questions  and  their 
corresponding  answers  would  not  fit  on  a  single  screen.  HyperWriter 
has  a  feature  that  deals  with  this  problem,  enhancing  the  use  of 
article-based  systems.  A  scroll  bar  automatically  appears  on  the  right 
side  of  the  screen  when  the  contents  of  a  node  are  longer  than  the 
number  of  lines  that  fit  on  a  screen.  The  scroll  bar  signals  to  the 
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user  that  laore  information  atretches  below  the  screen.  House  users  can 


click  on  the  scroll  bar  to  quickly  view  the  data.  Users  who  don't 
possess  a  mouse  strike  the  "Page  Dn"  key  to  perform  the  same  function. 
Overall,  the  feature  ensures  that  users  realize  that  more  information 
extends  below  what  is  currently  displayed. 

Oraanizational  or  Content  Nodes.  The  correct  combination 
of  organizational  and  content  nodes  was  sought  in  order  to  make 
navigation  through  the  document  quick  and  efficient.  The  final 
document  consists  of  ten  files.  One  entire  file  is  essentially  an 
organizational  node,  and  includes  a  "Master  File  Menu"  (table  of 
contents).  The  other  nine  files  correspond  to  the  nine  areas  of 
concentration.  Each  file  opens  with  an  organizational  node  that 
directs  the  user  to  standard  menu  choices.  Content  nodes  contain 
workbook  questions,  instructions,  and  information  about  the  Workbook . 

To  examine  the  actual  use  of  organizational  and  content  nodes  in 
greater  detail,  the  reader  is  referred  to  Appendix  C. 

Control  Panels.  Control  panels  offer  readers  connon 
navigational  commands  at  the  bottom  of  every  screen  (28:155). 
HyperWriter  does  not  support  control  panels  in  article-based  systems. 
However,  common  navigation  commands  were  positioned  at  the  bottom  of 
every  organizational  node  and  directly  after  each  question.  Ajppendix  C 
also  displays  the  standard  question  format  and  the  common  navigational 
comnands  for  each  question. 

Objective  4:  Testinx/Evaluation 


The  hypertext  system  was  tested  once  by  contracting  management 
students  from  the  Air  Force  Institute  of  Technology's  class  of  91S. 
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Ten  out  of  12  sttidents  responded  to  the  questionneire .  The  results  of 
the  questionnaire  are  located  in  /^pendix  D. 

The  results  reveal  that  the  majority  of  the  students  were 
knowledgeable  about  counter  operation,  and  had  no  problems  installing 
the  software  on  their  xonqmters .  This  should  be  representative  of 
individuals  who  would  desire  to  purchase  the  software  from  NCMA.  It 
seems  reasonable  that  individuals  wouldn't  purchase  diskettes  unless 
they  have  sufficient  knowledge  about  computers  required  to  install 
programs . 

All  of  the  students  agreed  that  the  cover  screen  provided 
adequate  instructions  required  to  activate  menu  choices.  All  of  the 
users  read  the  instructions  which  are  accessible  from  the  cover  screen, 
and  SOS  agreed  that  they  felt  confident  using  the  document  after  doing 
so.  With  the  instructions  that  were  available,  90S  agreed  that  a  paper 
users  manual  is  not  required. 

The  intention  of  the  researcher  to  provide  the  user  with  a  simple 
program  received  favorable  results.  All  of  the  students  reported  that 
the  system  was  easy  to  use,  and  90S  found  the  system  easy  to  learn. 

None  of  the  students  got  lost.  Menu  choices  and  displays  were  the  most 
popular  feature  of  the  program,  while  the  depth  and  breadth  of  data 
available  was  the  least  liked  feature. 

The  interest  of  the  program  is  hard  to  predict  based  on  the 
responses  to  the  question  regarding  the  preference  to  use  the  hypertext 
version  of  the  Workbook .  Several  students  have  never  seen  the  actual 
Workbook .  and  could  not  make  a  comparison  between  the  two. 


One  question  specifically  asked  whether  the  feature  named  Readers 
Notes  functioned  correctly  when  activated.  Readers  Notes  offers  the 
user  the  ability  to  add  their  own  notes  or  outlines  to  any  window  in  a 
hypertext  document.  The  researcher  experienced  problems  with  the 
function  during  the  development  process.  When  activated,  it  caused  the 
keyboard  to  lock  up,  and  the  conqmter  had  to  be  restarted.  Seventy 
percent  of  the  students  experienced  similar  problems.  For  this  reason. 
Reader's  Notes  were  not  incorporated  in  the  final  document. 

Several  students  wrote  general  conments  on  separate  sheets  of 
paper.  These  are  listed  in  Appendix  B. 

Obiective  5:  Implement  Feasible  Improvements 

Revisions  to  the  hypertext  system  wer^  performed  after  the 
questionnaire  was  administered.  The  improvements  primarily  focused  on 
standardizing  the  nine  areas  of  concentration,  and  correcting  spelling 
errors.  Direct  access  to  Reader  Notes  from  the  document  was  also 
removed  as  a  result  of  the  problems  discussed  above.  Feedback  from  the 
questionnaire  also  resulted  in  a  map  of  the  system's  structure  to  be 
provided  with  the  installation  instructions  in  order  to  give  users  a 
general  understanding  of  the  program's  layout. 

Simmarv 

This  chapter  presented  the  results  of  the  researcher's  efforts  to 
meet  the  objectives  set  out  in  Chapter  III.  Copyright  permission  was 
obtained  as  a  first  step.  Second,  HyperWriter  was  purchased  from 
NTERGAID  because  it  met  four  important  criteria. 
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The  third  phase  was  the  bulk  of  the  effort.  Once  the  researcher 
completed  an  initial  drawing  of  the  system's  structure,  the  entire  text 
was  converted  to  standard  ASCII  files.  After  hours  of  experimentation 
with  HyperWriter,  an  article-based  system  was  developed  that  uses  a 
combination  of  organizational  and  content  nodes,  in  addition  to  menu 
choices  that  are  similar  to  control  panels. 

The  hypertext  document  was  tested  in  accordance  with  objective 
four,  and  the  results  of  the  test/evaluation  reveal  that  the  hypertext 
program  was  easy  to  learn  and  use.  Improvements  to  the  document  were 
m^lemented  as  a  fifth  and  final  step  in  the  research. 


54 


Conclusions  and  Recommendations 


The  purpose  of  the  research  was  to  develop  and  test  a  hypertext 
version  of  the  CPCM  Candidate's  Workbook  and  three  Supplements .  Five 
objectives  structured  the  research  effort:  1)  Obtain  permission  to 
create  the  hypertext  workbook;  2)  Identify  the  important  features  of 
hypertext  authoring  systems  needed  for  the  research;  3)  Develop  the 
hypertext  model;  4)  Test  and  evaluate  the  hypertext  model;  and 
5)  Iiiq>lement  in^rovements  to  the  hypertext  system  that  were  identified 
during  the  testing  stage. 

Chapter  I  introduced  the  research  and  stated  its  specific 
purpose,  which  was  to  develop  and  test  a  hypertext  version  of  the  CPCM 
Candidate's  Workbook  and  Supplements.  Chapter  II  was  a  detailed 
discussion  of  professionalism  in  contracting,  and  issues  regarding 
hypertext.  Chapter  III  explained  the  five  objectives  that  defined  the 
methodology  that  was  used  to  develop  and  test  a  hypertext  workbook. 
Chapter  IV  used  the  five  objectives  defined  in  Chapter  III  as  a  forum 
to  answer  how  they  were  actually  carried  out  during  the  software 
development  effort.  Chapter  V  presents  the  conclusions  from  the 
research  and  suggests  recommendations  for  further  research. 

Conclusions 

A  complete  hypertext  model  of  cne  four  publications  was  developed 
by  the  researcher,  and  tested  by  AFIT  graduate  students.  The  research 
has  shown  that  an  electronic  form  of  the  CPCM  Candidate's  Workbook  can 
be  developed  using  a  hypertext  authoring  program.  The  evaluation 
demonstrated  that  individuals  can  install  the  hypertext  system  on  their 
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personal  conqmters,  and  quickly  learn  to  use  it.  The  cross  reference 
of  questions  to  their  corresponding  key  points  takes  less  than  a 
second . 

The  researcher  developed  the  hjrpertext  document  with  an  authoring 
system  that  was  chosen  based  on  a  number  of  inq>ortant  criteria.  The 
researcher  possessed  limited  experience  on  personal  computers,  and  was 
able  to  become  proficient  in  the  use  the  authoring  system  in  a  small 
amount  of  time.  Although  the  use  of  HyperHriter  was  conq>atible  with 
this  research,  there  are  a  number  of  other  software  packages  on  the 
market  that  offer  educational  discounts.  A  complete  analysis  between  a 
developer's  needs  and  the  features  of  a  system  should  be  made  before 
the  selection  of  future  systems. 

Based  on  this  research,  every  effort  should  be  made  to  dedicate 
time  up  front  to  the  design  of  a  logical  document  structure. 

Development  of  the  hypertext  document  did  not  occur  in  distinct  phases; 
however,  the  research  advanced  at  a  much  quicker  pace  once  a  coherent 
structure  of  the  document  was  completed.  The  time  required  to  complete 
a  hypertext  document  increases  at  an  alarming  rate  if  the  structure  is 
changed  late  in  the  development  process.  This  is  due  to  the 
impossibility  of  cutting  and  pasting  hypertext  links. 

A  document  should  be  analyzed  prior  to  an  attempt  at  converting 
it  to  hypertext.  The  font  size,  graphics,  underlines,,  and  severs  1 
other  factors  affect  the  ability  to  scan  text.  In  addition,  the 
highest  quality  scanner  should  be  used  during  the  conversion  process. 
Because  it  is  easy  to  underestimate  the  time  required  to  scan  and  edit 


documents,  sample  pages  should  be  scanned  and  edited  prior  to 
committing  to  a  development  effort. 

Reconnendations 

Three  recommendations  are  presented  here.  Two  describe 
possibilities  for  further  research.  The  first  reconmendation  is  to 
test  and  evaluate  the  hypertext  workbook  across  a  broader  population  of 
NCMA  members.  Potential  users  of  the  system  include  a  vast  population 
of  civilian  and  military  contracting  personnel  across  the  country.  It 
is  unknown  how  many  users  would  be  interested  in  a  hypertext  workbook 
that  operates  on  other  than  MS-DOS  operating  systems.  The  desire  for  a 
document  that  runs  on  Macintosh  computers  might  exist.  Predictions  of 
the  document's  usefulness  could  be  made  with  greater  confidence  if  the 
system  was  tested  across  a  greater  sample  of  the  population. 

The  second  recommendation  is  to  expand  the  depth  and  breadth  of 
the  data  available  in  the  current  system.  As  one  example,  unique 
contracting  terms  in  the  hypertext  document  could  be  linked  to  their 
associated  definitions.  Two  sources  of  terms  could  be  used  to  support 
this  effort.  The  first  is  the  Desktop  Guide  to  Basic  Contracting  Terms 
offered  by  NCMA  as  one  of  their  Educational  Product  Offerings.  The 
other  source  is  the  research  based  accumulation  of  acquisition  and 
contracting  terms  gathered  as  a  result  of  the  efforts  of  graduate 
students  from  the  Naval  Postgradu  ■  School  and  the  Air  Force  Institute 
of  Technology.  Such  an  effort  would  greatly  enhance  the  hypertext 
doctanent,  and  make  practical  use  of  the  two  sources  of  terms. 

In  accordance  with  /ppendix  B  (NCMA  letter  granting  copyright 
permission) ,  the  completed  hypertext  model  was  turned  over  to  the 
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Please  rate  the  questions  according  to  the  following  scale: 

1  -  strongly  agree 

2  **  agree 

3  -  neutral 

4  -  disagree 

5  strongly  disagree 
N  -  not  applicable 

1.  I  consider  myself  knowledgeable  about  general  computer  uperation. 

1  2  3  4  5  N 

2.  I  had  no  problems  installing  the  software  onto  my  computer. 

1  2  3  4  5  N 

3.  The  opening  screen  of  the  prototype  was  self  explanatory. 

1  2  3  4  5  N 

4.  The  opening  screen  instructions  allowed  me  to  activate  menu 
choices . 

1  2  3  4  5  N 

5.  I  read  the  program's  instructions  that  were  available  from  the 
cover  screen.  yes  no 

(if  no,  skip  to  question  6) 

a.  After  reviewing  the  instructions  I  felt  confident  using  the 
prototype.  1  2  34  5  N 

b.  With  the  instructions  that  were  available,  a  users  manual  is 
not  required.  1  2  3  4  5  N 

6.  This  system  was  easy  to  learn.  1  2  3  4  5  N 

7.  The  system  was  easy  to  use.  1  2  3  4  5  N 

8.  I  got  lost  (i.e.,  was  not  sure  where  I  was  within  the  document.) 

1  2  3  4  5  N 

9.  For  the  amount  of  time  spent,  I  could  learn  more  about  the 
exam  by  using  this  application  as  compared  to  the  paper 
version  of  the  Workbook. 

1  2  3  4  5  N 

10.  The  features  I  liked  about  the  system: 

(You  may  select  more  than  one  response) 

a.  displays  (use  of  color,  screen  organization,  contextual 

info) . 

b.  functions  (what  things  you  can  do  with  the  system). 

c.  menu  choices  (how  you  get  from  one  place  to  another). 
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d.  speed  of  processing  (how  fast  the  computer  does  things). 

e.  instructions  (on  how  to  use  the  system). 

f. .  data  (the  text  used  in  the  system). 

g.  depth  and  breadth  of  data  available. 

11.  The  following  features  of  the  system  need  to  be  improved: 

(You  nay  select  more  than  one  response) 

a.  displays  (use  of  color,  screen  organization,  contextual 

info) . 

b. '  functions  (what  things  you  can  do  with  the  system). 

c.  data  links  (how  you  get  from  one  place  to  another). 

d.  speed  of  processing  (how  fast  the  computer  does  things). 

e.  instructions  (on  how  to  use  the  system). 

f.  data  (the  text  used  in  the  system). 

g.  depth  and  breadth  of  data  available. 

12.  I  used  the  program  for: 

a.  less  than  30  minutes. 

b.  30  minutes  to  one  hour. 

c.  one  to  two  hours. 

d.  more  than  two  hours. 

13.  I  would  prefer  using  this  program  over  the  paper  version  of  the 
Workbook . 

1  2  3  A  5  N 

lA.  I  tested  the  prototype  on  a  computer  that  had  the  following 
features: 

a.  Processor:  8088/6  80286  80386  Don’t  Know 

b.  Nath  Coprocessor:  Yes  No  Don’t  Know 

c.  Display:  CGA  EGA  VGA  Don’t  Know 

d.  Memory:  6A0K  IMeg  other _  Don’t  Know 

15.  I  tested  the  READERS  NOTES  feature  of  the  program  yes  no 

(If  no,  skip  question  16). 

16.  The  READERS  NOTES  feature  of  the  program: 

worked  fine  caused  the  keyboard  to  lock  up 

17.  If  you  have  any  other  general  comments,  please  record  them  on  a 
separate  sheet  of  paper. 

(25:90-92) 
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January  8,  1991 


Capt.  Dean  McLain 
5797  Grots  Drive 
Dayton,  OH  45431 

Dear  Capt.  McLain t 

Tou  are  hereby  granted  pemltsion  to  create  an  Interactive  coaputerited  study 
tool  based  on  NCNA's  CPCM  Candidate's  tforkbook  and  its  Suppleaents. 

It  is  our  understanding  that  NCMA  vill  be  given  full  rights  to  reproduce  and 
sell  the  coaputer  diskette  and  user's  guide  resulting  froa  this  effort. 

Reproducing  all  or  portions  of  the  Workbook  and  Suppleaents  for  the  above 
purpose  vill  not  violate  NCMA's  copyright. 

This  product  vill  be  a  aajor  contribution  to  NCMA's  Educational  Product 
offerings.  Thanks  so  such  for  your  dedication  and  expertise  in  this  effort  I 

Sincerely, 


James  V.  Coggins,  CPCM 
Executive  Director 
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Appendix  D;  Questionnaire  Results 


The  following  scale  was  used  to  rate  the  questions: 

1  ~  strongly  agree 

2  -  agree 

3  -  neutral 

4  -  disagree 

5  ■*  strongly  disagree 
N  ~  not  applicable 

The  numbers  below  represent  the  percentage  of  people  (out  of  10)  who 
gave  the  indicated  rating. 

1.  I  consider  myself  knowledgeable  about  general  computer  operation. 

1  2  3  4  5  N 

30S  50Z  20% 

2.  Z  had  no  problems  installing  the  software  onto  my  computer. 

1  2  3  4  5  N 

70%  30% 

3.  The  opening  screen  of  the  prototype  was  self  explanatory. 

1  2  3  4  5  N 

50%  40%  10% 

4.  The  opening  screen  instructions  allowed  me  to  activate  menu 
choices . 

1  2  3  4  5  N 

60%  40% 

5.  I  read  the  program's  instructions  that  were  available  from  the 

cover  screen.  yes  no 

(if  no,  skip  to  question  6)  100% 

a.  After  reviewing  the  instructions  I  felt  confident  using  the 
prototype . 

1  2  3  4  5  N 

50%  30%  20% 

b.  With  the  instructions  that  were  available,  a  users  manual  is 
not  required. 

1  2  3  4  5  N 

40  50%  10% 

6.  This  system  was  easy  to  learn. 

1  2  3  4  5  N 

50%  40%  10% 

7.  The  system  was  easy  to  use. 

1  2  3  4  5  N 

40%  60% 
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8.  Z  got  lost  (i.e.,  was  not  sure  where  I  was  within  the  document.) 

1  2  3  4  5  N 

20X  30Z  SOX 

9.  For  the  amount  o£  time  spent,  I  could  learn  more  about  the 
exam  by  using  this  application  as  compared  to  the  paper 
version  of  the  Workbook. 

1  2  3  4  5  N 

lOX  30X  40X  20Z 


10.  The  features  I  liked  about  the  system: 

(Note:  The  individuals  could  select  more  than  one  response  for 
this  question.  Therefore  the  percentages  are  based  on  the  total  of  26 
responses  for  this  question.) 

19Z  a.  displays  (use  of  color,  screen  organization,  contextual 
info) . 


12Z  b. 
27X  c. 
12X  d. 
8X  e. 
15X  f. 
8X  g. 


ftinctions  (what  things  you  can  do  with  the  system) . 
menu  choices  (how  you  get  from  one  place  to  another), 
speed  of  processing  (how  fast  the  computer  does  things), 
instructions  (on  how  to  use  the  system) . 
data  (the  text  used  in  the  system), 
depth  and  breadth  of  data  available. 


11.  The  following  features  of  the  system  need  to  be  improved: 

(Note:  The  individuals  could  select  more  than  one  response  for 
this  question.  Therefore  the  percentages  are  based  on  the  total  of  12 
responses  for  this  question.) 


a. 

info) . 
17X  b. 
17X  c. 
17X  d. 
17X  e. 
8Z  f. 
25X  g. 


displays  (use  of  color,  screen  organization,  contextual 

functions  (what  things  you  can  do  with  the  system), 
data  links  (how  you  get  from  one  place  to  another), 
speed  of  processing  (how  fast  the  computer  does  things), 
instructions  (on  how  to  use  the  system) . 
data  (the  text  used  in  the  system), 
depth  and  breadth  of  data  available. 


12.  I  used  the  program  for: 

30X  a.  less  than  30  minutes. 

60Z  b.  30  minutes  to  one  hour. 
lOX  c.  one  to  two  hours. 

d.  more  than  two  hours. 

13.  I  would  prefer  using  this  program  over  the  paper  version  of  the 
Workbook . 

1  2  3  4  5  N 

lOX  SOX  lOX  lOX  20X 

14.  I  tested  the  prototype  on  a  computer  that  havi  the  following 
features: 
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a.  Processor:  8088/6  80286  80386  Don't  Know 


lOS 

30Z 

SOZ 

lOZ 

b.  Hath  Coprocessor: 

Yes 

No 

Don't  Know 

lOZ 

70Z 

20Z 

c.  Display:  CGA 

EGA 

VGA 

Don't  Know 

Monochrome 

80Z 

lOZ 

lOZ 

d .  Memory :  640K 

IMeg 

2Meg 

AMeg  Don ' 

t  Know 

30Z 

30Z 

lOZ 

lOZ 

20Z 

15.  I  tested  the  READERS  NOTES  feature  of  the  progrcm  yes  no 

(If  no,  skip  question  16).  lOOt 

16.  The  READERS  NOTES  feature  of  the  program! 

worked  fine  caused  the  keyboard  to  lock  up 

30*  70* 
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Appendix  E:  Additional  Comnents  from  Questionnaire 


1 .  Msnory 

a.  Seena  to  use  a  lot  of  memory.  I  ran  out  of  memory  the  first 
time  I  used  it  while  1  had  a  DOS  shell  and  some  TSR  programs  loaded  up. 
I  normally  have  enough  memory  to  run  most  software,  such  as 
Nordperfect,  Windows  applications,  Quattro  Pro,  DBase,  and  so  on. 

b.  Program  is  a  memory  hog. 

2.  Use 

a.  Would  it  be  possible  to  select  choices  by  number,  rather  than 
Bloving  a  cursor  around  the  screen?  I  found  that  a  little  laborious  at 
times. 

b.  Getting  around  in  the  program  was  not  hard.  What  was 
difficult  was  figuring  out  what  the  "big  picture"  was.  What  am  I 
supposed  to  with  it?  What  kind  of  answers  do  they  expect  on  the 
CPCM? 

c.  If  I  memorized  the  Key  Points  to  Consider  and  regurgitated 
them  on  the  exam,  would  I  pass? 

d.  Do  I  really  need  to  go  off  and  read  the  references?  If  I 
read  the  references  and  then  answered  in  my  own  word  in  essay  format, 
including  key  points,  would  that  be  all? 

e.  Is  there  any  way  to  move  to  a  specific  question  number  within 
a  BKklule  without  starting  from  the  first  question  and  highlighting 
"Next  Question"  until  I  get  there?  This  would  be  useful  if  someone 
does  not  do  the  whole  module  in  one  sitting. 

f.  Is  there  any  way  to  print  besides  using  print  screen?  It 
would  be  nice  to  print  the  Question,  Key  Points  to  Consider,  and 
Readers  Notes.  Overall,  it  looks  good. 

g.  I  like  it. 

h.  Why  does  pressing  Enter,  FI,  or  the  mouse  do  the  same  thing? 

■  ) 
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